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LT. INTRODUCTION 


mupomatic fault recovery iS the ability of a computing 
system to continue its specified logical performance after 
isolating failed physical components. This thesisS presents a 
simple recovery technique that incorporates system 
reinitialization in a real-time, distributed multiple 
microcomputer environment. The automatic recovery mechanism 
is designed specifically to support image processing 
applications where a record of previous computation is not 
required. The recovery mechanism uses a dynamic relocation 
algorithm as a means of reconfiguring the system as 
reinitialization from a standard initialization state is 
performed. 

miceaugomatic recovery system méchanism, developed by 
this thesis, is designed for a class of real-time systems in 
which the loss of a segment of data is tolerable. Because 
the loss of previous computations are not a dominant factor 
for recovery in this type of system, automatic fault 
recovery is simply a task of reinitializing the system and 
continuing execution. 

This thesis uses a flexible initialization mechanism 
designed by Ross [28] as the basis for an automatic fault 
recovery scheme based on system reinitialization. The 


reinitialization algorithm establishes a defined system 
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mpape (in particular that of the original initialization), 
with a different physical configuration. After 
reconfiguration, to eliminate faulty components, the 
reinitializaton mechanism allows the system to continue the 
performance of its logical prescribved tasks in a normal 


manner. 


A. FAULT TOLERANCE 

Automatic system recovery is part of a bvroader area 
entitled fault-tolerance. Although Unies thesis deals 
primarily with the concept of system recovery it is 
necessary to briefly identify and define the other areas 
Beat are included under the notion of fault-tolerance. By 
presenting a picture (or a model) of fault-tolerance, with 
mmemeiric rules relating to individual system requirements, 42 
clear and concise reasoning can te developed for automatic 
system recovery. 

Fault-tolerance is the architectural attribute of a 
cOmputer System that allows the System to continue it’s 
specific logical tasks when the system’s physical components 
suffer various kinds of failures. A fault-tolerant OC 
machine is capable of returning from an error state to a 
State of normal specific behavior thus assuring the survival 
of the information processing activities. Fault-tolerance 
consists of three sequential Steps: 


1. Fault Detection 


et 





2. Fault Diagnosis 
oe Fault Recovery 


Fault detection requires that the existence of a fault 
be realized. This is accomplished by a detection mechanism 
that observes some symptoms of the machine that indicates an 
error has occurred. Fault diagnosis takes place once a fault 
is detected. The error conditions are analyzed to isolate 
the fault cause. Steps are then taken to limit the adverse 
Brt+ects on the system and initiate the correct recovery 
measures. Finally, fault recovery involves specific actions, 
such as dynamic reconfiguration of the physical components, 
to secure continued system operation in a normal state or 
possibly a degraded mode dependine on the recovery mechanism 
implemented. 

The presence of fault-tolerance features in a system is 
a unique attribute. During normal (fault-free) operation 
fault-tolerance does not provide any performance advantages 
and in a fault-free machine would be superfluous. With the 
increase in technical knowledge, computing machines are 
becoming larger and more complex. AS fault-free devices are 
not a reality the the necessity of fault-tolerance in a 
computing system becomes more and more apparent. In the 
fault prone-physical implementation, fault-tolerance is the 
insurance of the logic machine against disruptive physical 


events [1]. 
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B. RECOVERY TECHNIQUES 

Recovery techniques are incorporated into systems in 
mraere to cope with failures. A failure is an event at which 
the system does not overform according to specifications. 
Failures can have numerous causes, but in a Computing 
system, most generally, are the result of either hardware, 
software or usé@r érrors. In order to deal effectively with 
failures additional componerts and algorithms must be added 
to tne system. These components and algorithms attemnt to 
ensure that faults , or occurrences of erroneous States, 
result in limited damage to system computations. Ideally 
they remove the faults and restore the system to a correct. 
state from which normal processing can continue. The 
additional components and algorithms required in a System to 
cope with failures are called recovery techniques cr 
mechanisms. 

Numerous recovery techniques have been developed, as 
there are many kinds of failures. The particular recovery 
mechanism employed in a computer system is dependent on the 
Meee, Of hardware a system uses, the software and data 
Structures involved, system applications and many more 
important individual system design characteristics. 
Consideration as to the degree and priority of system 
recovery is also necessary. Certain systems, such as missile 
trackine computers, must perform real-time recovery 


completely to a correct state , while a large data base 
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machine might be required to Tecover to a previous correct 
State thus only preserving the data in its files. In an 
isolated environment, such as an unmanned spacecraft, system 
recovery techniques might involve graceful degradation. In 
such a system, failed physical components and tne lack of 
Spares may require reconfiguration of the system in order 
for computation to continue in a degraded mode. Recovery 
mechanisms also encompass a degree of fault anticipation. 
Such techniques involve continued recording of data 
computations, or checkpointing , in order to have a recent 
correct state to recover to. Often redundancy plays a large 
role in recovery techniques where a system with a faulty 
physical component will simply switch to an identical 
component which is either performing in parallel or is a 
backup Spare. Many Systems, Such as nuclear reactor control 
systems, use a recovery technique that involves just a safe 
Shutdown once a serious fault has been discovered. 

No single recovery technique or series of recovery 
techniaues can cope with every possible fault. Many 
different kinds of recovery procedures have been developed, 
each technique with its own particular advantages and 
disadvantages, but each enabling a system to deal 
effectively with different kinds of failures in different 
environments. 

The recovery techniques considered in the followirg 


sections do not encompass all possible Schemes of automatic 
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fault recovery and are by no means the only categorization 
of recovery mechanisms. Instead some of the more widely used 
meenmnraues are discussed and the kinds of recovery they 
meordae, as related to real-time systems, are ovriefly 
described. 

1. Backup 

Automatic fault recovery incorporating a backup 
technique is designed to return thé system to a previous 
(presumably correct) state once a fault is detected and 
diagnosed. To accomplish this task the state of the system 
is periodically recorded. This recording or check pointing 
provides the most recent correct State of the system and 
establishes a point from which the system can be restarted 
and be expected to function normally if all faults have been 
meonrected. 

In real-time systems where execution times are 
Sretscal backup recovery provides 2 minimum restoration 
period when program functions are dependent on previous data 
computations. Additionally checkpointing, in conjunction 
with a backup recovery mechanism, is applicable in systems 
where data 1055S can not be tolerated. Depending on the 
extent of checkpointing, a copy of critical data can te 
continually maintained on auxilary storage and restored if 
necessary using an automatic backup fault recovery 


technique. 
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2. Reinitialization 


Reinitialization recovery mecnanisms are salvation 
programs [25] that restore the system to a valid state; that 
of the initialized system immediately prior to its original 
execution. Reinitialization recovery basically performs 
backup recovery to a permanently recorded system state (that 
of the mitia l system) without any facility for 
checkpointine. Because no data recording is done 
reinitialization techniques do oat provide for the recovery 
Of data Other than that provided during system 
mao badlization. 

Real~time systems that can tolerate intermittant 
losses of data are best Suited for the recovery techniaue of 
reinitialization. Data loss in such a system becomes simply 
a function of the time required for reinitialization. [In 
applications such as image processing the data loss is 
tolerable due to large amounts of relatively similar input 
marormation and the acceptable disruption in processing due 
to occasional faults [19]. 

cS. Redundancy 

Redundant recovery techniques employ multiple 
components or modules, to perform the identical task in 
parallel. The recovery mechanism is initiated 1 t. a 
disagreement occurs between modules at the end of task 
cOmputation. There are several basic approaches to redundant 


fault recovery, but all methods essentially involve the 
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Substitiution of a faulty module with one that functions 
properly. Hybrid redundancy [19] is a form of redundant 
recovery that involves a majority vote of the outputs of 
several modules. Disagreeing modules are replaced with 
spares (under control of agreeing modules) automatically. A 
similar approach termed duplex recovery [19] involves the 
comparison of the outputs of only two modules. I[f 
disagreement occurs diagnostic routines identify the faulty 
unit and it is replaced or disabled. 

The majority of real-time systems developed in the 
past, and especially those which operate in an isolated 
environment (no human maintainance available) have employed 
redundancy to some degree. Redundant systms provide the time 
response required for time-critical functions and because of 
pidir parallel computations data loss is usually not a 
result. The disadvantages to redundant recovery systems is 
realized in the overhead required to run identical multiple 
Systems. With tne increase in technical knowledge, real-time 
Systems are becoming larger and more complex. The additional 
em Ort and expense required to incorporate automatic 
redundant fault recovery techniques is often not desirable. 

4. Graceful Degradation 

Graceful degradation, or degraded recovery, returns 
the system to a fault-free state, but with a reduced 
computing capacity [1]. Graceful degradation often involves 


backup recovery or reinitialization to restore the system, 


17 





but faulty components are not replaced. 

Real-time systems, operating in an isolated 
environment, often employ a form of degraded recovery if 
Spares are not available or have been depleted. This form of 
recovery, involving reconfieuration of system components, 
allows a system to continue performing it’s normal logical 
tasks, but uSually at a reduced frdte. Recovery using 
graceful degradation can result in the loss of data if the 
nonreplaceable coOmponent is some form of memory. 

5. Safe Shutdown 

safe shutdown is the limiting case of graceful 
degradation {1}. It is carried out when the system computing 
capacity falls below a minimum acceptable threshold. This 
form of recovery is a fail-safe method that is employed 
usually as a last resort. Safe Shutdown allows a system to 
be halted before it causes severe camage to components or 
data and in some cases jeopardizes human life. 

The use of a safe shutdown scheme in a real-time 
system does not provide any significant advantages other 
than the avoidance of catastrophic consequences in a 
critical computing situation. Military weapons systems 
controlled by a real-time System would be an instance where 


safe shutdown might be employed. 
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C. MOTIVATION 

Tne Solid State Laboratory at the Naval Postgraduate 
School is presently conducting research in the area of image 
mmeeessing. Under the direction of Professor T.fF. Tao, 
research and development of ‘smart sensors for missile 
mueadance, radar, satellite Sirver rances dnd other image 
processing applications [22] is progressing. The smart 
sensor platform will require on-board data processing of 
large quantities of collected image data. To provide the 
required computing power to process this significantly large 
amount of data in freal-time, a multiple microprocessor 
system performing asynchronous parallel processinsge is being 
developed [2]. To control this computer system an operating 
System, using the Multics [16] concepts of Segmentation in 
conjunction with Reed’s [18] design of virtual processors, 
has been developed and is presently in the implementation 
stage. The basic microcomputer operating system design was 
developed by O°Connell and Richardson [15] and is based on 
the structure of a hierarchical s@curity kernel. O’Connell 
and Richardson provided a flexible operating system design 
that is fundamentally configuration independent and 
adaptable to a spectrum of systems. The real-time version of 
this “family” of operating systems was refined and 
implemented by Wasson [23] and Rapantzikos [17]. 

One of the primary goals of the Naval Postgraduate 


School project, directed toward development of a smart 
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sensor platform, is fault-tolerance. Dynamic reconfiguration 
within a multiple microprocessor computer syStem, due to 
periodic maintenance checks or failure eye specific 
components, is the basis for extended performance, if not 
survival in such a system. The ability of the smart sensor 
platform to detect faulty processors or memory Seements, 
diagnose the problems and then perform dynamic 
reconfiguration (if required) and automatic recovery is a 
necessity for the system in its Beoject@a, . Lsolated 
operating environment. 

The operating system design of Wasson its logically 
organized into a hierarchy that separates the user 
application processes from the kernel. This modular, layered 
design lends itself to dynamic reconfiguration where 
processes can be relocated among physical processors. 
Additionally the system initialization technique proposed by 
Ross [2¢] provides a bdasiS for an automatic recovery 
mechanism that will reinitialize the system on a2 new 
physical configuration after the detection of faulty system 


components. 


D. OBJECTIVES 
This thesis is intended to focus primarily on the area 
of dynamic réconfiguration and automatic ré€covery of a 


real-time, distributed, multiprocessor system in a 


fault-tolerant environment. Using the system initialization 
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mechanism design of Ross [28], as a basis for system 
reinitialization, and the Synchronization primitives 
developed by Wasson {23] and Rapantzikos [17], for process 
@e@erdimation, this thesis provides an automatic recovery 
mechanism specifically designed for a real-time, 
multiprocessor computing system. 

Fault-tolerant computer systems in the past have used 
fault detection and reconfiguration mechanisms which dealt 
with components at the level of simple devices such as 
flip-flops and adders. With todays LSI and VLSI technology, 
it is no longér appropriate to be concerned with such small 
meant ts. the unit of fault detection and reconfiguration 
should te on the scale of processor/memory [24]. 

In order to accomplish fault-tolerance functions on the 
processor/memory scale new methods of detection and recovery 
have been developed. Software controlled fault-tolerance is 
@ method that nas been successfully implemented in such 
experiental systems as SIFT [24], FTMP [3] and Pluribus 
{12}. Fault tolerance is accomplished as much as possible by 
programs in these systems rather than the conventional 
hardware methods traditionally used. This includes error 
Morrection, detection, reconfiguration and prevention of a 
faulty unit from having an adverse effect on the system as a 
whole. This modularization (processor/memory) of system 
Components allows fault detection to be based on modular 


performance. Detection becomes Simply an algorithm performed 
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by a system monitor that determines the correct functioning 
of a module. The monitor evaluation can tbe performed using 
various methods. In SIFT [24] a two out of tnree vote of 
processor/memory computation determines a faulty module. 
Recovery techniques in such a system consist of a monitor 
alzorithm that simply eliminates a failed module by marking 
it as faulty and replaces it with a Spare if available. It 
is the primary objective of this thesis to design a recovery 
technique that is software controlled. The use of Intel ‘s 
{SBC 86/12A Single Board Microcomputer with on board RAM 
provides the processor/memory module configuration necessary 
for such an algorithm-based recovery mechanism. 

Dynamic reconfiguration is usually encompassed in an 
automatic recovery scheme and essentially involves. the 
automatic reconfiguration of a system in order to eliminate 
the faulty components. Tne objective of a modular automatic 
recovery design, incorporating dynamic reconfiguration, can 
be realized based on the concepts presented by Schnell [21]. 
The ability to bind and unbind the physical resources to the 
logical resources of a syStem creates an environment 
Supportive of dynamic reconfiguration. This in conjunction 
with an automatic recovery technique, controlled primarily 
by the system software and designed specifically for a 
Teal~time, multiple microcomputer system, is the primary 
Objective of this thesis. 


several designs for system recovery have been developed 
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mimerecent years. Although specific techniques have been 
employed, enormous problems still remain to be solved for 
parallel orocessors and distributed processing [25]. It is 
the additional goal of this thesis to provide some solutions 
to the dilemmas facing fault recovery in parallel processing 
systems. 

The real-time, imaze processing project under 
development at the Naval Postgraduate School provides an 
enviroment that lends itself to a simple fault recovery 
meemmraue. Complete system re@initialization after dynamic 
reconfiguration is a feasible fault recovery method provided 
the time for system reinitialization does not significantly 
degrade performance. With the LSI and VLSI technology used 
mote image processing Environment the recovery time will 
not be a significant factor. Due to the enormous amount of 
continued input information a few frames not processed 
during reinitialization will result in only temporary loss 
of the image and will not significantly degrade performance 
i, 19). 

This thesis deals primarily with only one aspect of 
fault-tolerance, that of fault recovery. One must assume 
that fault detection and diagnosis have been performed prior 
to fault recovery and that the system recovery mechanism has 
been initiated as a result of a detected fault. It is on 


these assumptions that this thesis is based. 
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EB. THESIS STRUCTURE 

The introduction just presented is deSigned to provide 
meemeereader With a obvrief look at fault-tolerance as it 
applies to computer systems and in particular to the 
development decisions on which an automatic recovery 
technique is based. Chapter II will describe the hardware 
architecture of tne multiprocrssor system designated for the 
automatic recovery mechanism and the Support utilities that 
enhance the hardware performance. Cnapter III will provide a 
detailed account of system initialization and how the 
initialization mechanism was implemented on the system 
Demaware. Chapter IV will outline the dutomatic recovery 
design as it relates to the operating system and the 
nardware employed by the system. The final chapter presents 
conclusions and observations that resulted from this thesis 
memert dnd suggestions for further res@arch. Four appendices 
are also provided that give detailed descriptions of the 


system initialization programs and their implementation. 
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A. OPEPATING SYSTEM 

To use the musatiple microprocessor environment 
effectively for Teal~time image processing the application 
programs must be partitioned and distributed among the 
microprocessors. The operating system required to manage 
such a multiple microcomputer system must coordinate 
mover process communication and Syucnroni zation. 
Additionally the operating system is tasked with the 
management of system resources which include 1/0 and memory 
management. 

The distributed operating system designed by Wasson [23] 
and Rapantzikos [17] supports the multiple microcomputer 
Smeenonument. It provides control for a large number of 
asynchronous processes and is designed to manage the 
resources of a multiple microcomputer system. The operating 
System is structured as a hierarchy, supporting kernel and 
supervisor domains. Segmentation of memory [16] facilitates 
the sharine of inter-process data while at the same time 
isolating the address space of those processes that require 
no interference. The concept of virtual memory, where each 
process is provided with its own address space, as supported 
by segmentation, leads to a configuration independent 


system. 





Tne kernel manages all physical processor resources 
providing the user with an environment that is relatively 
hardware independent while the supervisor provides the 
interface between the kernel and application processes. 
Inter~process communication and synchronization is 
accomplished using eventcounts and sequencers [18] and to 
ensure expeditious handling of time-critical processing 
requirements 2 preemptive, priority scheduling mechanism is 
incorporated. 

The operating system is designed to control a group of 
multiprocessors which share a single system bus or possibly 
a set of up to four clusters of such microcomputers [22]. 
In order to limit the bus usage to a minimum, and thus 
provide increased performance, copies of the kernel are 
physically distributed to each microprocessors loceat 
memory. Thess allows for high-speed access to kernel 
functions without over-burdéening the shared system bus. 

eee distributicn of the operating system kernel 
Mecessitates its Execution by every processor. Thus the 
kernel design incorporates a scheduler that will allow each 
CPU to provide its own scheduling. This leads to an 
Operating system that has no concept of master-slave control 
but, is dependent only on system-wide synchronization 
variables to maintain system coordination and regulation. 

ie Tne kernel 


The kernel uses the concept of two-level traffic 
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Somtprol to manipulate system resources. Multiplexing of the 
physical processors amongst the more numerous virtual 
mmoeessors 1S accomplished by the Inner Traffic Controller. 
It is at this lowest level of the kernel that the hardware 
Of the physical machine is interfaced. At the higher level, 
the Traffic Controller, virtual processors are multiplexd 
among the larger number of partitioned application 
processes. At this upper level of the kernel the 
inter-process communication and Synchronization primitives 
are made available to the user application processes to 
solve the complex (application independent) System-wide 
synchronization of parallel processing. 
eee ne Supervisor 

In the multiple microprocessor operating system 
family, propose@ by O°’Connell and Richardson [15], the 
Mumervisor level of the system is designed not only to 
provide the kernel interface, but to support such functions 
as file mandégement. The modified real-time subset of this 
operating system family, implemented by Wasson [23] and 
Rapantzikos [17] for image processing, incorporates the 
supervisor only as a gate to the kernel. The supervisors 
gate is Simply an interface to the kernel for. the 
application process. The gate provides a single entry point 
to the kernel in which all user programs can access the 
Synchronization primitives. This allows tne supervisor level 


and application processes to be independent of the kernel 
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impleméntation details and maintains the hierachical desien 
of the system. 
Eee real-time Processing 

In the isolated environment of tne smart sensor 
platform, mea l= time processing ivory e Sa Li Mme=c hati cdl 
computations. Real-time systems must be controlled by 
Opeteatine systems that ensure time-critical processing is 
given immediate attention when required. 

The image processing programs of the smart sensor 
system are partitioned into Separate processes and 
meri puted among individual microcomputers. The ability of 
each processor’s kernel to schedule the image processire 
functions assigned =0 it is aecompersned by a 
meoority-driven preemptive scheduling technique which 
pPeoviaes for expeditious handling of processes which perform 
time-critical operations. Additionally the distribution of 
moc dpplication processes among tne physical processors 
local memories allows the same advantages as the 
distribution of the kernel. Performance is increased in the 
real-time environment by reducing system bus accesses for 
program ma Struc tions and data. The placement of all 
executable code and unshared data in local processor memory 
enhances the time=eritical processing required in a 


real-time system. 
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EB. HARDWARE 
1. Selection 


Pmeemicroprocessor Chosen to support tne real-time 
image processing project was the Intel &@@86. Sitenificart 
advantages over comparable microcomputers were realized in 
the final selection oat the @&@86 for the multiple 
microprocessor design. Performance Pree tcartones past 
experience with other [ntel products, and eSpecially the 
software and peripheral equipment support all added up to an 
off-the-shelf, immediately available microprocessor that 
Sumomsbe Sasily interfaced to the image processing project. 

2. The 8@86 Microprocessor 

The iogetwmeeeo 15 5a 16 bit, FMOS technology 
microprocessor. It has a 5 Megahertz (MEZ) clock rate and 
can address a full megabyte of primary memory. To provide 
high execution speed the S8@8&6 architecture incorporates 
instruction pre-fetch which allows for tne overlapping of 
Meerruction fetch and instruction execution cycles. 

The 8986 uses memory segmentation to divide the one 
megabyte of accessidle memory into logical units. A Segment 
Can range anywhere up to 64 kilo-bytes in length and can te 
placed anywhere within the one megabyte address space of the 
£086, provided the Segment base begins at a 16 byte boundary 
[4]. Although segmentation allows for the logical division 
of memory into an independent set of contiguous locations it 


must be empnasized that the segment boundry length is not 
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enforced by the hardware. Since the &@@86 does not Support 
explicit segment boundries, segments at the hardware level 
may be disjoint, partially overlapped or fully overlapped. 
momsupport the operating system, the design constraints must 
ensure segments of an individual process never overlap. The 
mechanisms to achieve this are presented ty Ross [28]. 

To obtain the effective address of a particular 
memory location tne 8@86 uses a base address and an offset. 
The base address must be a multiple of 16. In order to 
address the full megabyte of memory the 8@88E performs a left 
Sart of four bits on the base address, zero-filling the 
mauneeerower—-order bits. Once the base address has been 
shifted the address offset from the instruction counter 
register is added to the base value forming a 2e-bit 
effective address. 

The 8@&6 processor has direct access. to four 
segments at any one time [4]. Their tase addresses are 
contained in four segment registers depending on the segment 
use. Tne Code Segment (CS) register contains tne base 
address of the code segment from which instructions are 
Meenecd. The Instruction Pointer (IF) register provides the 
offset from the CS value to the next executable instruction. 
The Stack Segment (SS) register maintains a pointer to the 
base of the Stack segment. The Data Segment (DS) register 
contains the addrees of the current data segment and the 


Bxtra Segment (ES) register provides an additional Seement 
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maemess that is typically used for external or shared data. 
3. The iSBC 86/12A Single Board Microcomputer 

ime 1SBC G€6/12A is a complete microcomputer platform 
Bele it contains a S5MHZ @@&6 processor, S2 kilo-bytes of 
random—-access memory (RAM), @ kilo-bytes of electrically 
programmable read-only memory (EPROM), programmable serial 
and parallel I/0 interfaces, a programmable a Ger ow 
controller, a real-time clock and an interface to the Intel 
Multibus for interconnneéction to other devices [11]. 

The iSBC 86/12A provides the basic hardware support 
Beguarea fOr a multiple processor operating system. The 
Multibus interface provides each processor with the ability 
to independently access a globdal Shared memory Segment. The 
8@86 processor provides a built-in semaphore instruction 
Which allows individual CPUs to set a lock on the system 
bus, and thus control global memory access. The iSBC S86€/12A 
also can be confisured to provide preempt interrupts 
(between processors) by connecting the parallel I/O ports to 
the Multibus interrupt lines. Finally the EPROM can 0be 
programmmed to contain tne bootstrap program that will 
intialize the system. | 

4. Intel MDS Development System 

Program development for the real-time multiple 
microprocessor project waS accomplished uSing the Model 232 
Intellec Series II Microcomputer Development System (MDS) 


(4). The hardware and software support provided by tne MIS 
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meeea sienificant factor in the original choice of Intel’s 
2986 CPU and iSBC 86/12A single board computer for use in 
the system. 
a. Hardware 

peconderyestorage for the multiple microcomputer 
System was not available and therefore the MDS system with 
mise tloppy disc file storage, as shown in Figure II-1, was 
used to simulate secondary storage for the iSBC 86/12As. 
This was particularly important during system initalization 
Gpaeere:nitielization. Since the Multibus was not connected 
to secondary storage all disc accesses were accomplished 
through the single iSBC 86/12A connected to the MDS via a 
serial port link. System I/O was coordinated by a bootstrap 
meeerdm in the case of initialization or by a run-time 
mummers process during system execution. Essentially the iSBC 
@6/12aA connected to the MDS was required to execute a loader 
meeeess, when disc I/0 was ré€équired, loading data into a 
global memory buffer. The other single bcard processors 
could then accomplish their individual memory loading by 
Meeesosing the global memory buffer. It snould be noted that 
this simulation of secondary storage by the MDS is only 
Mmemered until a hard disc is installed and interfaced to 
the Multibus. 

bee SoOrtware Utilities 
The MDS software support provided by the 


manufacturer waS again one of the prime considerations for 
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the selection of the Intel products used in the multiple 
microcomputer system. The utility programs provided were 
merece mpensively in tne system generation phase to create 
the operating System and the initialization programs. 

Tne PL/M-86 compiler [7] provided the necessary 
Support to allow syStem programming to be accomplished in 
tne flexible, hign-level language of PL/m-86 [5]. The 
language is totally reenterant aS reenterant code is 
Seenpial for the kernel code that is snared by the user 
processes. fhe PL/M=86 compiler offered four modes of 
meeetton that allowed the programmer to select the degree 
of segmentation during translation. The compact mode of 
compiler operation was used primarily during the system 
Benerdtion as it afforded the most flerible use of the 
segmented address space during process relocation. 

The LINKS86 [(6} utility program was used to 
combine the separately developed and compiled program 
feemeeeS into a single, ré€locatable otject module. fhe 
linking ability provided by this utility routine allowed the 
programmer to develop small manageable program modules that 
could be debugged and maintained separately and then teund 
into a Single module prior to loadine. 

The Loces6 [6] support program produces an 
absolute object module from the input relocatable object 
meee, This utility routine provides tne programmer with 


the ability to locate object modules at any location in the 
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one megabyte of addressable memory space. 

Finally OH86 [6] was used to convert an object 
female tO a hexadecimal, ASCII formatted, object file. This 
utility program provided formation of an object module in 
nexadecimal, that could be easily manipulated once loaded 
into primary memory. The format of the hexadecimal file was 
Such that a Simple program within the kernel could read and 
relocate temob ect fite. The same program of the xernel 
also converted the hexadecimal module back toa binary 
object module. This was necessary in order to allow normal 
Ptecution of the file. 

CeminesisBC 957A-iSBC 66/128 Interface 

Tne iSBC 9574 Intellec-iSBC @6/12A Interface and 
Execution Package [9] contains the hardware and software 
required to interface an iSBC 86/12A Single Board Computer 
with the Intellec Microcomputer Developement System (MDS). 
Recall that the system bus (Multibus) that is used by the 
iSBC &86/12As was not connected to any sort of secondary 
storage. In order to simulate secondary storage for the 
System one of the iSBC &6/12As was connected to the MDS and 
the iSBC 957A interface package I/O routines were used to 
access the MDS floppy disc drives. 

The iSBC 957 A interface package contains 
software utility programs that were used extensively in the 
Mmescearch and developement environment of tnis thesis. The 


iSBC 957A package system I/O routines interface with the 
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ISIS-I1l operating system running on tne MDS. The routines 
can be activated by PL/M-86 high level language procedure 
_ummemmmewiere tie 158C 9S7kA procedures are declared external 
in the PL/M-86 program. This allowS programs executing in 
the iSBC 86/12A to perform I/0 with the MDS floppy discs. 
Additionally the iSBC 95748 interfaces with the iSBC §86/12A 
monitor providing the use of the monitor commands for 
program debugging on the iSEC 86/124. ! 

An iSBC 957A system I/0 procedure is first 
Called in the bootload phaSe of SyStem initialization. The 
bpootload program calls the routine LOAD [9] to load the 
bootstrap program, stored on disc, into a buffer in main 
global memory. This allows all the remaining processors 
access to the bootstrap routine. The LOAD process requires 
five parameters to be passed to it. The first argument 
passed iS a pointer to an ASCII String containing the name 
of the file on disc to be loaded. The next parameter passed 
to the LOAD routine is a word containing the value of zero; 
this argument has no effect as it serves only as a 
placeholder. This parameter is followed by a word that acts 
Osa SWitch. This argument 1s set by the programmer and 
im@icates that control be either returned to the calling 
program or that contol be transferred to the program just 
loaded. The next argument is a4 pointer to a pointer in whicn 
the starting address of the loaded program is placed. The 


meoal argument passed to LOAD is a pointer to a word in 
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men the monitor can place 2 status code indicating a 
nonfatal error has occurred during the LOAD routine. 

The iSBC S5S7A system 1/0 procedures are also 
meeae in the bootstrap process of syStem initialization. 
During the bootstrap program tne OPEN, READ and CLOSE [9] 
routines are called to read a hexadecimal object file 
memeaining the base laver of the op@€rating system into a 
buffer in global primary memory. The OPEN procedure locates 
muemeemecified file to be read, on disc, and then initializes 
ISIS-II tables and buffers in the Intellec system. Five 
mememeters are passed to the OPEN routine. The first 
argument iS a pointer to a word in which the monitor stores 
the active file transfer number (AFTN). This number is used 
to identify the file to other iSBC 957A SyStem Me 
mmumereaures. The next parameter is a pointer to an ASCII 
String containing the file name. Following the pointer to 
meemrr.e€ name is a word containing the access mode for whicn 
the file is being opened. This argument identifies the file 
Seeeeaoute as read, write or read and write. The next 
parameter is a word containing a tile number that is used 
only if line editing is taking place (this argument was not 
used). The final argument is a pointer to a word in which 
the monitor could pass @ status code if a nonfatal error 
occurred during tne OPEN routine. 

MiecmCAh  procedcuress iS @called by a PL/M~86 


program to transfer up to 4896 bytes of data from an open 
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file to amemory location specified by the calling program. 
The first argument passed to READ is a word containing the 
active file transfer number (this will be the same file 
Humber dssiened in the open procedure, if OPEN and READ are 
used in conjunction). The next parameter iS a pointer to a 
feeere tO which data of the open file is to be transferred. 
A word containing the number of bytes to be transferred is 
the next paramenter passed to READ. This argument is 
followed by a pointer to a word in which the actual number 
of bytes transferred is placed upon completion of the REAL 
procedure. The final argument passed to READ iS a pointer to 
a word in which the monitor will return a status code in 
event of a nonfatal error during READ routine. 

A call to the CLOSE procedure will cause the 
ISIS-II operating system to delete the tables and tuffers 
that were allocated wnen tne specified file was opened. The 
arguments that are passed to CLOSE include the word 
containing the active file number (the same as assigned in 
OPEN) and a pointer to a word in which the monitor can 
return a status code snould a nonfatal error occur during 
the CLOSE routine. 

The only otner iSBC 957A procedure used was the 
EXIT (9) routine. This procedure allowed a FL/M-86 program 
memereing on the iSBC 86/12A to return to the monitor if it 
was called. The EXIT routine was used only for program 


development and debugging. 
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Although the iSBC 957A system I/O routines were 
also used in the run-time loader process to load the 
application processes and by the ioader process in tne 
operating system for system reinitialization it must be 
emphasized that the iSBC 9574 package was used only to 
Simulate an environment. The lack of a hard disc for system 
secondary storage necessitated the use of the iSEC S57A 
software and hardware to simulate the required auxilary 
storage. Future plans for syStem design (See Figure I[I-2) 
meerude the conrection of a hard disc to the Multibus for 
secondary storage. When Chis Ocemrs the Simulated 
environment will be eliminated as will be the requirement 
for the iSBC Q957A-iSBC &86/12A Interface and Execution 


Package. 
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Ci. oe oe oie LAG IZATION 


A. DESIGN 

System initialization is the method used to get an 
operating System loaded and runnine on a computer system. A 
simple system initialization mechanism has been designed by 
Ross (2¢] that can be used with a variety of hardware and 
operating system configurations. During system initalization 
Ross outlined three phasSeS that must be accomplished, 
sequentially, in order to get an operating system loaded and 
running on a computer system. First, a core image of tke 
operating system is created. This is known as system 
generation time. It normally is done on a Separate 
development computer system and consists primarily ef 
developing the operating system and initialization code. The 
next phase of initialization is bootload time. This is the 
point where the lowest level of the operating system is 
a@ctudily loaded into the primary memory and its system 
parameters and tables are initialized. finally when the 
operating system programs are running normally tne 
initialization sequence is considered to have entered the 
run time phase. 

mesinitialization M@Cchanism involves three separate 
loading functions as Shown in Figure [III-1. The bootload 


program runs on bare system hardware, during bootload time, 
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and is used to load into global memory a bootstrap program. 
This program is ROM-resident so that it may be activated by 
a bootload switch. The bootstrap program, loaded by the 
bootload program, also runs on the bare system hardware and 
will be used to load tne base layer of the operating systen 
into primary memory and Start it running. The final loading 
function is part of the distributed operating system and is 
loaded into each processor during the bootload phase along 
with tne base layer of tne operating system. This loader is 
used during run time to load the remainder of the operating 
system and tne application programs dnd to prepare them _ to 
meescheduled and run. 

Implementation of Ross” system initialization design was 
the first effort of this thesis with the premise that the 
imitialization technique would be the basis for system 
Bemmitielization. This se@ction deals primarily with the 
specific implementation of the initialization design as it 
applies to the operating system of Wasson [23] and 
Rapantzikos {17} and the Intel iSBC €6/12A Single Foard 


pc rocomputer. 


B. SYSTEM GENERATION TIME 

The development of the operéting system and 
initialization tasks takes place at system generation time. 
mms is the first step of initialization and takes place 


prior to the bootload and execution phases. Pregram 





development during system generation was accomplished almost 
entirely on the Intel Microcomputer Development System 
(MDS). The use of the ISIS-II operating system in the MDS 
system with, its supportive utility programs, provided a 
flexible environment in WHICH to accomplish system 
generation tasas. The complexity of the bootload ard 
run-time phases was Significantly reduced by use of the MDS, 
in conjunction with the ISIS-II operating system, to 
compile, link, locate and debue programs during the system 
generation phase. 

In the initialization design by Ross (20), several 
assumptions were made at system generation time that greatly 
Simplified bootioad and run time development. Although some 
of these assumptions will not hold in the following chapters 
concerning automatic recovery techniques, for the purpose of 
System initialization alone this discussion will make the 
Same initial asSumptions that Ross does. These asSumptions 
permit extensive preliminary processing to be done in the 
more flexible atmosphere of SyStem generation thus relieving 
later phases, wnicnh occur in much 1é€sS Supportive 
environments, of the preparatory processing that they would 
Otherwise be required to perform. 

The key assumption at system gen@€ration time is that the 
initial hardware and software configurations are known. This 
allows initial memory allocation decisions to re 


accomplished (prior to loading and execution) in the 
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supportive atmosphere of the Intel MDS. The significance of 
knowing the initial configuration is realized in the ability 
of the system developer to allocate memory on a ge#lobval cr 
Mecal scale. Ae was pointed out in the section descritire 
the operating system, it is highly desirable to place as 
many programs in local memory as possible in order to 
Climinate bus contention. Only shared, writable segments 
snould be allocated to global memory. 

System generation is viewed as a sequence cf events, 
beginning with program design and ending with tne creation 
of the load module or core image to be loaded. This thesis 
wd 1 concentrate on the Speci: ue implementation 
considerations of the initialization scheme rather than the 
design methodology. A detailed examination Ot system 
generation events and the choices made throughout the 


development of the initialization design is discussed by 


Ross [28]. 


C. BOOTLOAD TIME 

The system initialization mechanism was designed to 
commence operating once a “pootload switch was activated. 
moe in turn causes a jump to the first instruction of tne 
dootload program which is contained in read-only memory 
(ROM). The bootload program is a Small Simple program that 
runs on the bare hardware and is Moca Led in each 


microcomputer’s ROM. The bootlcad program serves two 





purposes. It’s primary function is to load a ‘bootstrap 
program from secondary storage (i.e., a hard disk) which 
Meeeretoen be executed to continue the majority of system 
maetialization. Proceeding in this fasnion allows the 
ROM—resident bootload program to remain small and relatively 
Simple. Secondly the bootload program serves to uniquely 
identify each physical processor. Each microcomputers copy 
of the bootload program differs only in that it contains a 
unique serial number that identifies the physical processor. 
This unique processor number is placed ina global CPU 
more, during execution of the bootload program, and will be 
used by the bootstrap program to identify the physical 
processors during the remaining phases of system 
initialization. 

A time~Ssequence of activities takes place during 
bvbootload time, beginning when the bootload switch is 
pressed, and ending wnen tne operating system kernel is 
moeded and running. In this particular system the operating 
system, as was described previously, is distributed to each 
single board computer and therefore must te loaded into eacn 
computer’s local memory. Therefore, each microcomputer’s 
dbootload program must be activated as Lt is the 
responsibility of each individual CPU to load its own system 
programs. Activation of all tne processor bootload programs 
can be accomplished simultaneously using a simple bootloaed 


Switch that is connected to all CPUs. 
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a7. System Activation 
In the implementation e¢escribed by this thesis, 


using one to eignt iSBC 86/12A single board microcomputers, 
it is necessary to indicate to every iSEC @6/12A when to 
begin executing the ROM bootload program. Jgalbhe Was 
accomplished during development in the form of a simulated 
bootload switch. In the experimental environment the INTR 
button on the iCS &8@ Chassis [(1¢] served to simulate the 
bootload switch. Depressing this button places a hardware 
interrupt on the system Multibus which can be received by 
all iSBC 86/12AS plugged into the iCS 8@ Chassis. Interrupt 
number two is the Multibus interrupt line activated by 
pressing the INTR button. All iSBC 86/12AS can be jumpered 
to acknowledge this interrupt by wiring tne incoming 
Multibus interrupt line (post E71) to the 8@86 non-maskable 
interrupt line in the carne matrix (post E89) [11]. Note 
that to make the non-maskable interrupt active, the ground 
wire (between post E87 and E89) must be disconnected. Figure 
ite shows the correct iwiring. The non-maéskable interrupt 
on the 8@86 has been used to start the system initialization 
mechanism due to the disabling of the maskable interrupts 
when the iSBC 86/12A is in tne monitor. The initialization 
routine commences with all boards, except the MDS-connected 
iSBC 86/12A (as noted below), in tneir respective monitors. 


Only the non-maskable interrupt iS capable of interruptine 
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the 8886 CPU in this State. 

When all iSBC &86/12A boards have their interrupt 
matrix modified as outlined above it iS possible to commence 
the bootload phase, causing all SBC 86/12A°s to execute the 
bootload program, load the operating system kernel, ard 
commence Kernel execution, by Simply pushing the INTR buttor 
on the iCS 8@ Chassis. The bootload program is the interrupt 
handler. The four byte non-maSkable interrupt vector, that 
will be loaded with the address of the entry point to the 
bootload program, is the third interrupt vector in the 
interrupt table [4] (interrupt 2; address ¢86@:22098 to 
@GGG8:00GB). Activation of the non-maskable interrupt on the 
8086 causes an unconditional, indirect jump to the tootload 
program via the non-maskable interrupt vector. 

System design calls for the bootload program to be 
ROM-resident, but to facilitate debugging in the 
experimental environment, it was located in RAM. During this 
development period the iSBC 86/12A monitor command, LOAD 
{9}, was utilized to download the bootload program from the 
meoe Llioppy dise prior to activation of the initialization 
mechanism. Recall that only one iSBC &6/12A was connected to 
the MDS in this simulated environment, thus allowing only 
that particular single board computer to be loaded using the 
monitor LOAD command. This in turn, required that the 
bootload program, once loaded, be placed in all the 


remaining iSBC 86/12As by the monitor MOVE [9] command as it 
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was impossible to load the individual iSBC 86/12A°S memories 
directly. Additionally, all interrupt vectors were required 
to be preset to the bootload program entry address before 
the initialization routine could be activated. 

Finally the MDS-connected iSBC &6/12A was required 
to have exited it’s monitor before the non-maskatle 
interrupt would function properly. This requirement was the 
result of MDS interference during the interrupt sequence. To 
meeeethe iSBC 66/124, connected to the MDS, of it’s monitor 
ot was necessary to Stari the seveo™ CFU. executing 
maeeructions from RAM. The program executed for this purpose 
was in the form of a2 loop at the beginning of tne bootload 
module. When interrupted the CPU then functions identically 
momtne remaining processors. Note that all the other iS85C 
86/12As were interrupted while in their respective monitors 
and functioned normally, thus they required no looping 
mechanism. 

It is necessary to emphasize that the above Sequence 
mame vents is required only in the experimental environment 
when placing the bootload program in RAM. When the debugged, 
final version of the bootload program is located in EPROM 
the steps involved above will not be applicable. 

2. The ROM-resident Bootload Program 

The bootload routine is a small, simple program that 

will be EPROM resident (See Appendix B). The first function 


of the bootload process is to determine the Footload CPU. 
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The Bootload CPU will serve as the master or controlling CPU 
throughout the bootload and run time loéding pnases. wWnile 
the bootload programs in all CPUs are identical, the 
Bootioad CPU will execute some sequences of instructions 
that the otner processors will not. The PL/M-86 language 
provides a built in procedure known as Lockset [5] that 
permits to programmer to implement a software lock (viz., a 
busy wait). This procedure uses a variable located in global 
memory to control the bus access. In order to designate the 
Bootload CPU, a deliberate race condition is entered as all 
processors begin executicn of the bootload program. Bach CFU 
attempts to set a software lock, using 4 global variable 
(CPUSTBLSLOCK), and then enter a table in global memory 
Known as the CPU Table (CPUSTABLE), shown in Figure III-3. 
The built in procedure Lockset with it’s global parameter 
(CPUSTBLSLOCK) is used to resolve tne conflict of multiple 
Simultaneous access attempts to the CPU Table. Thus only one 
meareat da time can access the CPU Table and the’ first CPU to 
do so becomes the Bootload CPU. 

After entering tne CPU Table (CPUSTABLE) eacn 
processor will fill in entries in the table and then unlock 
the bus to allow the other CPUs access. The CPU Table is 
indexed according to logical CPU numbers where tne Bootload 


CPU is deSignated @. The next CPU to get control of the bus, 
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after the Bootload CPU, and enter the CPU Table, becomes 
logical CPU 1 and so on. 

Once a processor has gained control of the bus using 
tne global bus lock variable (CPUSTBLSLCOCK), and accessed 
the CPU Table (CPUSTABLE) the first action performed is for 
the CPU to enter its serial number (CPUSID). Recall that 
this serial number is different for each ROM-resident 
bootload program and tnat this number uniquely identifies 
every physical processor in the eeu Next a counter, 
(CPUSTOTAL), 15 incremented in order for the Bootload CPU to 
keep track of the number of physical processors present in 
the system. Lach CPU is identified additionally by a logical 
CPU number, (LOGSCPUSID), that identifies it, aS mentiored 
berore, according to its sequence of entry into the CPU 
Table. The next set of instructions executed in the bootload 
program increments @ logical CPU number (LOGSCPUSNUM). This 
globdal variable will be used by the next processor, to eain 
access to tne CPU Table, and will serve as an index into the 
CPU Table. Finally the sottware lock on the system bus is 
released and the identical sequence of entries into the CPU 
Table iS performed by the next processor to gain access to 
the bus. This continues vntil all physical processors have 
accessed the CPU Table -and made the appropriate entries. 
Upon completion the CPU Table (CPUSTAELE) will contain eecnh 
individual processors unique serial number (CPUSID) entered 


according to the sequence of CPU Table access. This allows 





the processor to be identified by a logical, as well aS a 
physical, CPU number. Additionally tne Bootload CPU will 
have recorded the total physical CPUS it counted in the 
system in it’s own CPU total (CPUSTOTAL) field in the CPU 
Table. Note that the CPU Table contains a mailbox (CPUSMAIL) 
entry and an acknowledgement (CPUSACK) entry for each 
processor. These entries in the CPU Table will be used later 
in the bootstrap program for system synchronization. 

After completion of the above sequence the FBootload 
CPU will execute another PL/M-86 built-in-procedure called 
TIME [5]. This untyped procedure causes a time delay in 
multiples of 188 microseconds based on a & MEZ clock and the 
Geg86 CPU cycle time, without interruptions. In the bootload 
program the Bootload CPU will execute a time delay of 12 
Mmepeaseconds. This delay will allow all the other processors 
maemetime necessary to access tne CPU Table before the 
bootload CPU commences its actual loading action. 

Tne hardware configuration for system development, 
as described in the hardware section, allows for only one 
iSBC 86/124 to be connected to the MDS (using tne iSBC 
957A-iSBC 86/12A interface and execution package). This 
means that only the single board CPU with this connection 
can access the disc files. This Simplifies the bootload 
programs by eliminating the need for a complex 
Synchronization method to allow tne processors to snare the 


disc, but neccessitates a controlling or Eootload CPU to 
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serve as the main access to disc files for all CPU’s. 
Because the Intel hardware dictates this pan tc wlar 
configuration, it is necessary to designate the &6/12A 
memeste board microcomputer connected to the MDS, and thus 
the disc files, as the Bootload CPU’. In order to default 
the particular processor with the MDS connection as the 
Bootload CPU a time delay has been added to the instructions 
of the bootload procedure, BOOTLOADSINTR (in the bootload 
program), of all CPU’s except the MDS connected iSBC &6/12A. 
This added time delay in all the processsors, except the 
Bootload CPU, is executed as the first instruction upen 
entering the bootload program, thus allowing the iSBC 86/12A 
connected to the MDS to access tne CPU Table (CPUSTABLE) 
first and become the Bootload CPU. It should be emphasized 
that tnis and the unique physical CPU numter are tne only 
difference in tne bootload programs loaded to the various 
physical processors and is dependent on the nardware 
configuration. Note that with a hard disc, Serving as 
secondary storage, connected directly to the Multitus (i.e., 
all processors are capable of disc access) the need for the 
default delay will be eliminated as any CPU can serve as tne 
Bootload CPU. 
5S. Bootstrap Program Loading 

The next function of the bootload pregram is to load 

@ bootstrap program. The bootstrap program (see Appendix C) 


contains the actual instructions that will load the bese 
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layer Of the operating system. Ey performing the 
initialization in this sequence, the bootload routine 
remains small and the primary egoal, of a Simple EPROM 
resident bootload program is achieved. 

The hardware configuration, as described in the 
previous section, allows for only one iSBC &6/12A to be 
Connected to the MDS and necessitiates tnis CPU to be the 
Bootload CPU. Because the Bootload CFU is the only processor 
Mumemeecan access the disc files, it must load the files 
containing the Bootstrap program and the operating system 
into Blobal memory buffers and then allow the other 
individual CPU’s to execute or load the files as required. 

Tne bootstrap program is loeded by the Bootiload 
CPU using a 957A I/0 procedure called LCAD [9]. As was 
previcusly described in tne hardware section, this utility 
procedure requires that five parameters be passed to it. The 
mere argument is a pointer to an ASCII string of tne file 
name of the file to be loaded. In this case the bootstrap 
program (BTSTRP). The next parameter, known as tne bias. is 
not used for this implementation. Following this is a 
Derameter called the switch. This is set to allow tne LOAD 
procedure to return to tne bootload program. The next 
memeument 15 &@ pointer to the starting address of the loaded 
program (BTSTRP) which is assigned to the variable 
Bob ISTRESADR. The last pramenter passed iS a4 Status 


variable for error codes. The Bootstrap program’s location 
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in global memory iS predetermined at SyStem generation thus 
the bootstrap program loaded using tne iSBC 957A LOAD 
procedure isa file created by LOC86 which iS in executable 
format (viz., not a hexadecimal file.) 

Having successfully loaded the Eootstrap program 
meoe £il0bal memory the Bootioad CPU will transfer control, 
with an unconditional jump, to the starting address of the 
Bootstrap program. This :transfer of control takes pléce 
using a PL/M-@6 Indirect Procedure Activation [5] (i.e., 
simply a call with a pointer). The iSBC 9574 LOAD procedure 
automatically placed the start of the bootstrap program in 
the start address parameter (STSBISTRPSADR) when it loaded 
the Bootstrap program. The call, using this bootstrap Start 
address (STSBTSTRPSADR), simply sets tne CS and IP registers 
of the Bootload CPU to the Starting address of the bootstrap 
meoeram, puts tne parameters to te passed, LOGSCPUSID, tne 
address of CPUSTABLE and the address of CPUSTBLSLOCK, on the 
Stack and then executes an unconditional jump. This 
transfers control from the EPROM bootload program in the 
Bootlaod CPU to the bootstrap program just read in from 
disc. 

While the Bootloed CPU is erecuting the instructions 
to load the bootstrap program, the remaining processors must 
ma Ler a wait state. Since the bootload programs are 
executing on bare hardware the Operatine System 


synchronization mecnanisms are not available. Tne solution 
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to CPU synchronization has been to implement a software 
Spinlock in the SPROM resident bootload program called 
CPUSWAIT. This procedure allows all CFU’s except tne 
Bootload CPU to wait in the Bootload program until they are 
instructed by the Bootload CPU to transfer control to the 
vootstrap program. The indication for a particular CPU to 
jump to the bootstrap program, as tne Bootload CPU did with 
a pointer call, will be the placement of the bootstrap start 
address in the CPU“s mail box. Once the processor sees it’s 
mailbox no longer contains the initialized null value it 
will transfer control from its own EPROM bootload program to 
the bootstrap program. Note that the bus lock must be set 
each time a particular CPU accesses The CPU Table 
(CPUSTABLE), in the ocr procedure CPUSWAIT, and then 
released when the CPU exits. This allows tne spinlock to 
function normally in all CPU’s with every processor getting 
a chance to check its mailbox periodically. If tnis werent 
the case one CPU could lock the bus and enter a permanent 
wait state (in CPUSWAIT). With the bus locked tne Bootload 
CPY would be unable to gain access to the CPU Tatle 
(CPUSTABLE) to signal the processor in tne CPUSWAIT 
procedure to transfer control to the bootstrap program. The 
result would be a deadlock condition. 
4. Bootstrap Program Execution 
The bootstrap program, created at system generation 


time, will load the base layer (kernel) of the operating 
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system from disc into primary memory (see Appendix F). As 
outlined in the previous discussion concerning the operating 
System, the kernel will be distributed to all physical 
processors and thus each processor will need to ereécute the 
bootstrap program to load it’s individual kernel. The 
Bootload CPU, now executing in tne bootstrap program will 
coordinate the kernel loading among processors and will also 
Mmomtroe actual disc access for all CPUs. 

The actual entry point to the bootstrap module is 
the procedure BOOTSSTRAP. Since the bootstrap program is not 
linked to the bootload program the address of the procedure 
BOOTSSTRAP must be the Start of the bootstrap module. The 
entry point must be a procedure as the transfer of contrel 
from the bootload program to the boostrap program is a 
procedure call (ie., call by pointer) which passes 
parameters. The parameters passed are required by the 
Boovload CPU to maintain control of the initialization in 
the bootstrap program. The parameter LOGSCPUSID identifies 
Seem processor as it enters the bootstrap program. fhe 
parameters containing the address sof CPUSTABLE and 
CPUSTBLSLOCK (pointers) are used to address tased variables 
[5], CPUSTAEPLE and CPUSTBLSLOCK, which function identically 
as they did in the bootload program. 

The first action of the Bootload CPU, in executing 
the bootstrap program, will be to read into 4a globel memory 


buffer (KERNELSBUFFER) the hexadecimal file containing the 
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base layer of the kernel. This i5 accomplished using, as was 
previosly described in the nardware section, the iSBC 957A 
Interface Package System I/O procedures [9] in conjunction 
with the ISIS-II operating system. The first procedure 
called is OPEN [9]. This procedure eSsentially locates the 
kernel file on disc and assigns to it an active file 
transfer number (KERNELSAFIN). The next ISBC 957A procedure 
called is READ [9]. This routine identifies the open file by 
its active file transfer number (KERNELSAFITN) and then reads 
a maximum of 4996 bytes from disc to the global memory 
buffer (KERNELSBUFFER). After doing so READ returns. the 
number of bytes transferred in the word TRANS and updates a 


e 


file marker according to the number of bytes actually 
Memmererred. The Bootload CPU will continue to erecute the 
{SBC 957A READ procedure in the bootstrap program until the 
bytes transferred are less than tne maximum bytes allowed 
for transfer (4996) indicating the end of file has been read 
and loaded into the kernel buffer (KERNELSBUFFER). Finally 
the procedure CLOSE [9] is called allowing the ISIS-II 
operating system to perform the actions necessary to close 
the file with the previously asSigned active file transfer 
number (KERNELSAFTN). 

The kernel file just read into the kernel buffer 
(KERNELSBUFFER), by tne Bootload CPU, is a hexadecimal file 
created during system generation time by CH86 [6]. When the 


kernel file is transferred to tne kernel buffer it reméins 
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in its hexadecimal format. The procedure HEADSHEXSFILE will 
convert the hexadecimal object file (the kernel) into its 
binary (exe@€cutable) representation and load it at the 
address Specified in the hexadecimal file. READSEEXSFILE is 
executed by the target CPU to load the kernel into it’s 
local memory after being signalled to do so by the Bootload 
CPN. This method of loading the kernel file as a hexadecimal 
file was used due to the documentation available, by Intel, 
with respect to hexadecimal data records. Ross [2G] also 
provides a detailed explaination of hexadecimal record 
mermat. Documentation concerning binary object files was 
less clear than the hexadecimal documentation and did not 
Drovide for e@asy rélocation during tne bootstrap loadire 
sequence. 

Since the Bootload CPU was the. first processor to 
transfer control to the bootstrap program and is the only 
processor executing in the bootstrap program at this point, 
it calls the procedure READSHEXSFILE as soon as it has 
completed loading the kernel file and passes to it the 
address of KERNELSBUFFER. READSHEXSFILE now loads the kernel 
file located in global memory into the local memory of tke 
Bootlocad CPU. Note that the location of the kernel file in 
local memory is determined at System generatior time. 

All other processors are still executing the EPROM 
bootload program, waiting to be Signalled by the FEootload 


CPU via tneir respective mailboxes . Tne Bootload CPU will 
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determine the number of remaining processors waiting to load 
the kernel file by setting the Bootload CPU (logically 32) 
processor count equal to the total CPUs (TOTALSCFUS) minus 
one (the Bootload CPU doesn’t count itself’). The Bootload 
CPU now signals each CPU in turn to load its kernel 
(converting hexadecimal to object) and then waits in a 
Spinlock until that particular processor has completed that 
Portion of the bootstrap program that loads the kernel into 
local CPU memory. The signal placed in the target CPUs 
mailbox is just a pointer to the procedure BOOTSSTRAP (in 
global RAM) which allows the target processor to identify 
Pmemstart of tne bootstrap program and transf€r control to 
that address with a pointer call. 

The syStem initialization mechanism is designed to 
handle kernel files that differ according to individual 
CPU’s assigned functions. For this reason the Bootload CPU 
allows only one CPU to load the kernel at a time. This 
febows the Bootload CPU to check which CPU a particuléer 
kernel is targeted for and then send the appropriate signal 
for loading. If the kernel loaded for all processors was 
identical then the Bootload CPU could signal all the 
remaining CPUs, simultaneously, and the loading of the 
kernel could proceed’ in parallel. Note that in the 
particular implementation used for development dy this 
thesis the kernel loaded was identical for all CPUs, but the 


loading was accomplished sequentially tO fremain consistant 
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with tne overall design. 

As in the bootload program the bootstrap routine is 
executing on bare hardware and thus no synchronization 
mechanisms are available for process coordination. To 
provide process synchronization a Spinlock identical to that 
used in the bootload program was implemented. The procedure 
WAITSCPU allows the Bootload CPU to enter a wait state after 
Signalling @ particular processor to transfer to the 
bootstrap program and load its Pete When the target CPU 
has completed loading its kernel it signals the Bootload CPU 
via the acknowledge flag (CPUSACK) in the CPU Table 
(CPUSTABLE). The Bootload CPU then continues to the next 
logical CPU and repeats the Slenalling action until all 
processors, as indicated by the total Cru Count 
(TOTALSCPUS), have loaded their respective kernels. 

As each processor completes its bootloading task it 
will enter a wait state by calling the procedure CPUSWAIT. 
mem CPU will remain in this wait state, executing a 
Spinlock, until all processors have completed their 
respective bootloading tasks. When the loading of the kernel 
file has been completed by all processors the Bootload CPU 
will signal all CPUs to perform an unconditional jump to the 
Start location in their respective kernels. This is 
accomplished by the Bootload CPU setting the acknowledge 
flag (CPUSACK) for the Bootload CPU in the CPU Table 


(CPUSTABLE). 





Since the kernel is not linked to the bootstrap 
program tne transfer of control from the bootstrap program 
to the kernel iS accomplished by an indirect procedure 
activation (viz., a call by pointer). During tne previous 
execution by all CPUs of the procedure READSEHEXFILE, where a 
kernel was loaded into each CPU’s individuél local memory, 
the Code Segment (CS) and Instruction offset (IP) were 
obtained 10-5 SochmerGa 1G Udi ake cnet, 8) Mem Gs dnd amr 
constitute the entry point (Start address) of each 
particular CPUs kernel. 

A bootstrap pointer variable (MEMSKCSIPSPTR) is 
employed using the PL/M-86 language AT attribute [5] to 
perform the necessary transfer of control to the kernel. The 
AT attribute locates a two word structure (KCSIP) at the 
address of the pointer variable (MEMSKCSIPSPTR). Effectively 
this allows tne four byte location in memory reserved for 
the pointer variable (MEMSKCSIPSPTR) to be accessed two 
bytes (a word) at a time. Immediately prior to tre call by 
pointer (using MEMSKCSIPSPTR) the first word, of the two 
word structure, (ZCSIP.SEG) is set equal to the kernel code 
Segment CGS.) that was determined by the procedure 
READSHEXSFILE. The second word (KCSIP.OFF) is set to reflect 
the kernel instruction pointer (IP). Since the two word 
structure (KCSIP) uses the identical location in memory as 
the bootstrap pointer variable (MEMSKCSIPSPTR) the result is 


to establish the Kernel entry point in the bootstrap pointer 
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variable. This allows a pointer call (using MEMSKCSIPSPTR) 
to transfer control from the bootstrap program to the start 
of the kernel module. 

The pointer call will also pass parameters to tne 
kernel, 1 Yai particular the logical CFU identification 
(LOGSCPUSID) and the poysical CPU Re Mmei: Lodi! or 
(PHYSSCPUSID). These arguments are required by the kernel 
meoeesses in order to identify individual processors. The 
transfer of control to the kernel is executed by all 
mmoeessors, including tne Bootload CPU, after tne Bootload 
CPU has Signalled that the loading of the kernel is 
complete. 

It is necessary to keep all processors in a wait 
mayer in the bootstrap program and transfer control to tke 
kernel in mass. Should CPUs be allowed to jump directly to 
meear particular kernels immediately after completion of 
kernel loading, but prior to completion of kernel loading by 
all CPUS, the global shared variables used by the kernel 
could be, and most probably would be, altered. These shared 
variables are “loaded as part of each kernel, and 
therefore, would revert to their initialized values. The 
global shared kernel variables provide for process 
Synchronization and inter-communication and require the 
presence of all CPUs and respective processes, assigned at 
System generation time, to function correctly. Allowing 


processors to transfer intermittently to their kernels would 
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lead to improper initialization of the operating System and 


erroneous execution. 


D. RUN TIME 

meee transfer of control from the bootstrap program to 
the kernel, by eacn physical processor in the system, marks 
Meeme termination of the bootload phase and the Start of the 
Fun—-time phase of system initialization. During run-time all 
the user’s application processes will be loaded from 
auxiliary storage by a kernel process called the run-time 
meager. Unlike the bootload and bootstrap programs, that 
meres required to execute on the bare hardware of tne system, 
the run~time loader will be supported by the kernel 
functions to facilitate synchronization during tne loading 
of the application programs. 

1. The Kernel Interface 

The entry into the kernel requires that the 

parameters passed from the bootstrap program (LOGSCPUSID and 
PHYSSCPUSID) be fYremoved from the stack and that the 
environment of the kernel be eStablished to ensure proper 
performance of tne operating system. This is accomplished by 
a Special kernel interface set of instructions called the 
intialization sequence (see Figure III-4) that is located in 
the Inner Traffic Controller (ITC) Scheduler module [23] of 
mee kernel. 


To simplify the transfer of control the entry point 
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; FILE SKED.ITC 


sESTABLISH STACK STRUCTURE FOR PASSED 
sPARAMETERS FROM THE BOOTSTRAP FRCGRAM 


STACK-STRUCTURE STRUC 


RETURN DD Sa? 
PARM2 DB ? 
XXX2 DB ? 
PARM1 DEV 
XXX1 DB ? 
STACK-STRUCK ENDS 


PRDS SEGMENT EXTERNAL 
sRESERVE MEMORY IN THE KERNEL FOR TEE 
sPARAMETERS PASSED FROM THE EOOTSTRAP 
7PROGRAM 
LOGCPUID DEB ? 
PerscreulD DE 7? 

PRDS ENDS 


»BEGIN THE ITC SCHEDULER SEGMENT IN THE KERNEL 
SCHEDULER SEGMENT 


sBEGIN THE KERNEL INITIALIZATION SEQUENCE 
s;ESTABLISH THE BASE OF TEE STACK-STRUCTURE 
MOV BP,SP 


,SET UP STACK USING BP AS A BASE POINTER AND 
;STORE THE PARAMETERS PASSED FROM TRE BOOTSTRAP 
7 PROGRAM 

MOV CL,(3P) .PARM1 

MOV ES :LOGCPUID,CL 

MOV CL, (BP) .PARM2 

MOV ES ;sPHYSCPUID,CL 


;sJUMP TO THE KERNEL INITIALIZATION PROGRAM 
JMP KERNEL-INIT 


s;CONTINUS WITH NORMAL ITC SCHEDULER CODE... 


KERNEL INITIALIZATION SEQUENCE 
Figure III-4 
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maco the kernel is the start address of the ITC Scheduler 
module. All processors will execute the initialization 
Sequence, at the Start of the ITC Scheduler, once transfer 
from the bootstrap program is complete. The start of the 
mamtpadiization Sequence is in effect a special entry point 
into the kernel which is used for initialization only and 
thus executed only once. All other entries to the ITC 
scheduler consist of calls to specific procedures witnin the 
module, and therefore, never encounter the initialization 
sequence. 

The first set of instructions in the initialization 
Sequence will allow the parameters passed from the bootstrap 
program (LOGSCPUSID and PHYSSCPUSID) to te popved off the 
present stack and Stored under identical names reServed in 
the kernel’s Processor Data Segment (PRDS) [17]. The PRDS is 
@ per processor data Segment that will be utilized by the 
Memoel for specific processor TdenPecicati on. Having 
completed the transfer of parameters from the bootstrap 
program, the initialization sequence will tnen jump to a 
special initialization program [17] to establisn the correct 
Seeeucion @nvironment for the kernel. The initialization 
program is tasked with initializing the kernel data 
Structures. Specifically the initialization program will 
memsoe the idle process to be initialized to running ana the 
kernel loader process will be reflected as ready in the 


Virtual Processor Map (VPM) [23,17]. Once the proper kernel 
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environment has been established, normal kernel execution 
Can commence. This just requires a transfer On -contne b) from 
the seectal Initialization proaram to the kernel ITC 
scheduler that then schedules the loader process, since it 
is on the highest priority, ready virtual processor. 

2. The Run-time Loader 

The Run-time Loader is a Kernel process that will be 
employed to load the application pregrams from secondary 
Storage. Because the Loader process has a higher priority 
than the Idle process (the lowest priority- always) and 
Since no other processes are yet defined in the system, the 
jump to the ITC Scheduler at the end of the bootload phase 
appears to the kernel aS a preempt interrupt of the idle 
virtual processor. This preempt causes the higner priority 
Loader process to be scheduled and run on each physical 
processor. 

The kernel Loader process will have the benefit of 
the operating system primitives provided ty the kernel. In 
particular the ITC Advance and Await [23] procedures will 
provide for process Synchronization and communication during 
the loading sequence of the application processes. 

The details of the Run-time Loader process will be 
postponed until the next Chapter since a significant portion 
of the mechanism is incorporated in the automatic recovery 
mame. Once the concepts of system reinitialization have 
been presented in Chapter IV, the kernel Loader process will 


ve described in detail. 
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IV. AUTOMATIC RECOVERY DESIGN 


This chapter presents an automatic recovery design that 
is based on SyStem reinitialization. The mechanism for 
system initialization, described in the previous chapter, 
nas been modified to form an automatic system recovery 
routine that integrates with a hierarchical, distributed 
Meemating system to support fault-tolerent operation. First 
@ brief overview of the design is presented and then a 
detailed description of the automatic system recovery 


mechanism is described. 


A. DESIGN OVERVIEW 

Automatic recovery begins once a system has detected and 
diagnosed a component failure. [It is the responsibility of 
Qan @rror routine (for tne purpose of this iscussion 
encompassing both error detection and diagnosis functions) 
Mmemeraacate the particular component that has generated the 
system failure. Once the failure has been isolated, by the 
identification of it’s source, it iS then the recovery 
mechanism’s responsibility to perform the operations 
necessary to return the system to a normal, fault-free 
state. 

The automatic recovery technique employed in this desien 


results in a complete ré€initialization of the system 
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establishing a predefined initialized state. Upon completion 
Oreine automatic recovery routine, the system’ will have 
returned to a state identical to that of the original 
dvootstrapped system and will be prepared to begin normal 
execution. Many of the techniques used for automatic system 
recovery were previously employed in the initialization 
Mmomeane described in Chapter III. For this reason it is 
possible to incorporate the automatic recovery mechanism 
with the initialization routine to provide an overall desien 
that includes voth system initialization and automatic 
system recovery. 

System initialization and automatic recovery perform the 
same baSic functionss that of complete system restoration. 
For initialization the restoration of the system begins from 
a cold Start with the activation of a bootload switch, 
while the automatic recovery process is initiated by an 
error routine to restore or reinitialize the system. AS 
Figure IV-1 shows, after initialization or automatic 
recovery has commenced the basic tasks performed are 
identical. First a bootstrap program is invoked, executing 
on the bare system hardware, to load the kernel. This is 
followed by a transfer of control from the bootstrap program 
to the kernel where an operating system loader routine will 
be engaged to load Thema bp plavecdt }Onmememrocesses. The 
distinction between the initialization sequence of events 


oma, that of the automatic system recovery routine is based 
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Meeeune fact that initialization is executed only once, 
establishing the system configuration for the first time, 
while automatic recovery involves continued reconfiguration 
and reinitialization for tne lifetime of the system. 

The contrast between initializing the system for the 
first time and subs@quent reinitialization during automatic 
recovery is distinguished by the potential loss of system 
components, due to incorrect performance, during automatic 
system recovery. Additional tasks must be employee durire 
reinitialization, that are not applicable during 
mmepraltization, to compensate for tne loss of system 
components. These tasks must specifically deal with system 
mecontiguration and process rélocation in order to return 
the system to an initialized state that will allow continued 
normal, fault-free performance. 

Gomplete reinitialization involves reloading, from 
auxiliary stoTfage, all system processes from the lowest 
level of the operating system to the user’s application 
programs. The requirement for complete fYreloading of the 
system results from the fact tnat all modules are physically 
connected by a primary, shared bus (the Multibus [4]) ard 
migeereulty component can potentially affect all system 
modules and cata. The automatic recovery mechanism is 
mestened to deal with faulty components on the module level 
of processor and local memory. Specifially the design calls 


for the use of the iSBC 86/124 Single Enard Microcomputer to 
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be employed as the system component that Welel 2 be 
reconfigured during System reintialization. 

Elimination of a particular module during automatic 
meeeeen recovery, due to incorrect or faulty performance, 
will require that the individual processes which were 
assigned to that module be relocated. The 1058S of a module 
as a result of automatic system recovery will require 
reloading of the system processes on a new hardware 
configuration, Lhucmebac tinea tne Serernitidna zat ton routine 
with memory Management during process reloading and 
merocation. 

The real-time fYrecovery tasks developed in this design 
can be expanded to afford fault-tolerance to a wide Spectrum 
fee multiple computer systems. The flexible system 
environment created throueh the use of dynamic 
mecontiguratlon supports a variety of multi-processor 
functions. The concepts involved in the automatic recovery 
mechanism provide the basis for fault-tolerent computing ty 
allowing continued normal system operation after the 


Seimination of faulty components. 


B. RECOVERY INTERFACE 

Once automatic system recovery commences the 
fault-tolerence routines involving error detection and 
diagnosis are assumed to have been completed. As was aleuded 


to previously, this thesis does not attempt to identify any 
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specific error routines. [t is of no consequence to tne 
recovery mechanism how errors were determined, only that 
they have been diagnosed. Although specific error detection 
mechanisms are immmaterial to the automatic recovery 
routine, it is necessary for the iaterface between the 
Beutines to Encompass communication and Serene tion in 
order to estatlish a Smooth transistion into the recovery 
Foutine. The interface to the recovery mechanism is the 
responsibility of the error routine and serves the purpose 
of establishing a predetermined, consistant system state 
that will always allow automatic system recovery to proceed 
Correctly each time the routine is invoxed. 
im ne Error Routine 

Mice section. briefly outlines the error routine 
requirements necessary to support automatic system recovery. 
As was previously mentioned, it is beyond tne scope of this 
thesis to develop the specific error routine mechanism. This 
Section should serve only as a possible example for future 
development of the error procedure. 

The system error routine is required to establish a 
previously known system state for the interface into the 
recovery process. This State will Simply be defined as the 
state of the system prior to loading (tootstrapping) tne 
“system processes. Additionally the error routine will be 
required to have performed it’s defined task} ta tort 


eliminating the faulty module. In this design, that will 
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entail nalting the faulty processing module (iSEC 86/124) <so 
that is can no loneer participate in system execution. 

The error routine is assumed to te executing on all 
modules once a fault is detected. An error routine diagnosis 
mmoeram will then determine the faulty module. This could te 
as the result of a two out of three vote or a test program 
that indicates the faulty module. In any caSe the Specific 
faulty module is identified. 

Since the improperly functioning module has been 
mreeviously determined, the @€rror routine is simply required 
to halt the faulty processing unit and then initiate the 
recovery process. The operating system's preempt interrupt 
provides a relatively straight-forward way for the error 
routine to eliminate a faulty module. First the error 
routine will establish the idle process [22] as the highest 
priority process capable of execution on the faulty 
processor unit. This is just a matter of altering the 
priority in the faulty CPU’s Virtual Processor Map [23] 
causing the virtual processor dedicated to the idle process 
to be the highest priority. Then the particular processor on 
which the error routine is executing must send a preempt 
Signal to the faulty processor module that will force the 
malty module to run the idle process. This will effectively 
make the improperly performing module unavailable to any 
Merer processes. The idle process, running on the faulty 


module, will then be required to check a system wide error 
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moore, indexed by logical CPU numter. to determine if a halt 
should be executed. The error routine will have previously 
Sopecne halt flag for the faulty processing unit and the 
result will be the elimination of the failed module from 
participation in system execution. 

Additionally in the event the faulty module has 
failed completely (i.e., the CPU is unable to execute the 
idle process), the error routine is tasked with physically 
disabling the module Prom the system. This can be 
@ccompliished by incorporating in the error routine a 
hardware disable’ mechanism that will eliminate the faulty 
module from system interaction. 

emce the €rror routine has eliminéted tne faulty 
module from the syStem it will perform a sequence of tasks 
Miereeewill establish the interface environment for the 
automatic Tecovery mechanism. Specifically the error routinre 
Mee be required to reinitialize the Configuration Table 
(see Figure I7-2) and then transfer control to the bootstrap 
program. The Configuration Table is a modified version of 
the CPU Table designed to support both initialization and 
reinitialization and will be employed by the bootstrap 
program in the same manner as described in Chapter III. 

a. The Configuration Table 

The Configuration Table is a global frecord 
structure that will be used primarily to record memory usage 


and CPU availablity during automatic system Trecovery. As 


on 
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shown in Figure IV¥-3, three basic structures comprise the 
Configuration Table. The first, called the CPU Total, will 
Dey r€initlelized by the error routine to reflect the number 
of fault~free processors available to the system at the time 
of automatic recovery. Because the error routine has 
Knowledge of the total processors in the system prior to 
automatic system recovery, either from the initialization 
meaueeme Or from a previous execution of the automatic 
recovery process, it can determine the number of properly 
Mmimev1oning modules to enter in the CFU Total structure 
after performing elimination of the faulty module. 

The next structure in the Configuration Table is 
a multiple entry record that is indered by logical CPU 
Mumber. “he first fields in this structure are identical to 
the same CPU Table fields described in Chapter III. The 
error routine will be responsible for reinitializing the 
unique physical processor Serial numbers for each fault-free 
processor in the system. This essentially involves allowirg 
each processor to access the Configuration Table, one at a 
time, to enter it’s CPU identification number much in the 
Same fashion as the processors were numbvered in the bootload 
program during system initialization. As in the pbootload 
program the logical numbering of the CPUs in thke 


Configuration Table is performed in a random manner. 
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The Configuration Table will also contain a CPU 
mailbox and a CPU acknowledge entry for each logical 
processor in the system. These entries will be used during 
the bootstrap program for CPU synchronization as was the 
case in the dootstrap program described previosly. Note that 
the CPU Table used for system initialization in Chapter III 
memeancorporated in aoe Contigureareon stable. slals allows the 
system initialization routine to use the Configuration Table 
Seructure in the same manner as the CPU Table and provides 
compatibility between the initialization programs and the 
munomatic recovery routine. 

Additionally, the Configuration Table will 
include a 1l0cal, per processor memory map anda global 
memory map that will te used to support the memory 
allocation mechanism used foe reinitialization. TO 
Bec rlitate the recording of memory vwsage during automatic 
recovery, PeroLy has been logically subdivided into pages of 
256 bytes in length. The global and local m@€mory maps in the 
Mericuration Table are bit mars that will reflect the 
memory utilization of the system as reloading of the system 
processes proceeds. Speci facet, each processor will 
represent it’s 32 kilotytes of local memory using a 16 byte 
bit map. As shown in Figure IV-3, a 16 byte array is 
associated with each logical processor number in the 
Configuration fable structure. Additionally the global 


memory map, shown in Figure IV-, will consist of a 5&4 byte 
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array which will allow tne memory allocation mecnanism the 
Capability of accounting for the one megabyte of addressable 
memory minus the possible eight module local memories. Note 
that although the module memory of each iSBC 86/12A can bde 
divided between local and global memory the real-time system 
design dedicates all iS2EC 86/12A memory (32 kilobytes) to 
local memory to be used by the @¢G86 CPU. As a result no 
global memory will reside on any of the iSEC &6/12As. This 
means that all global memory will be provided by separate 
dedicated memory boards. | 

Beem cOutte=mration Table 1S a static Structure 
that is created at system generation time based on the 
maximum number of modules to be employed in the system and 
maemmaximum amount of memory to be utilized. Once the error 
routine has zeroed all entries in the Confieuration Table, 
tnen entered the total CPUs available to the system in the 
CPU Total field and reintialized all the processor’s unique 
ID numbers, it will be required to reload tne bootstrap 
program. 

db. The Load CPU 

The Load CPU serves as the coordinator of the 
automatic recovery routine, performing Similar duties as 
that of the Bootload CPU described in Chapter III. The title 
of Load CPU is assigned to the first CPU to access the 
Mentiguretion Table during tne reinitialization of the 


unique physical processors Serial numbers. The Load CPU is 
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moeecal CPU number zero in the Configuration Table. Since 
Maes reinitialization of the physical processor numbers is 
accomplished in a random fashion, any one of the fault-free 
CFUs remaining in the system is capable of being the Load 
feu . 

Moc TemroreroOutineGuwlll tdsa.ene Ofodd © CPU = wi th 
the job of reloading the bootstrap program into elobval 
memory. Recall that as in Chapter III, the primary tesk of 
the bootstrap program executed during automatic System 
mecovery, is to load the kernel. 

2. Recovery Activation 

fioeeserror routine will activate dautomatic system 
recovery by allowing the Load CPU to transfer control from 
m@iemerror program to tne bootstrap program it just reloaded 
into global memory. All remaining processor modules will 
fumempead wait state in their respective error programs. Note 
Mecsuethis sequence of events is identical to the action that 
took place Wiel the bootload program ior system 
mmeptalization. All CPUs, except the Load CPU, will enter an 
active spinlock in their respective error routines waiting 
por a4 signal from the Load CPU in the form of the bootstrap 
address, before transferring control to the bootstrap 
mmomram. The error routine wait state is the consistant 
state all processors (except the Load CPU) will enter durirg 
the recovery routine interface and is the state from which 


system reinitialization will always commence. 
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The Load CPU will transfer control to the just 
loaded bootstrap program using an indirect procedure 
activation (viz., a call by pointer) in the same fashion as 
the Bootioad CPU did in system Tratiel Eze t lone The 
Parameters passed to the bootstrap program will include a 
Beomter to the Configuration Table, a pointer to a global 
bus lock variable that is used to control access to the 
Configuration Table and the logical processor identification 
mumoer., Once the Load CPU has transferred control to the 
bootstrap program and passed the parameters just descrited, 


automatic system recovery will commence. 


C. OPERATING SYSTEM BEINITIALIZATION 

Automatic system recovery commences from a predetermined 
state established during the interface to the automatic 
recovery routine. The purpose of this defined state is to 
create a consistant environment from which the 
reinitialization process can always begin correctly. The 
previous discussion described the interface state that wes 
determined by the error routine. [t is in this state that 
Mmeemtirst part of reintialization, that of the kernel, 
begins. | 

The reinitialization of the kernel is accomplished using 
a bootstrap program that performs the identical tasks as the 
bootstrap program descrited in Chapter III. All processor 


modules, under the control of the Load CFU, will have tke 





opportunity to execute the global bootstrap program in order 
to load their respective kernels. Once the Load CPU has 
transferred control from the error routine to the bootstrap 
program the actual process of reinitialization will begin. 
1. The Eootstrap Program 
The primary task of the bootstrap program is to 
reload the kernel. The first processor to enter the gelobal 
bootstrap program will be the Load CPU. Recall that all 
remaining processors are waiting in their respective error 
routines until the Load CPU signals it is their turn to 
transfer to the bootstrap program and load their individual 
mornels., 
a. Kernel Reinitialization 

The distributed kernel is reinitialized by the 
bootstrap program which loads eacn processor module’s (iSBC 
86/12A) local memory with the required kernel processes. The 
bootstrap program will perform identically to the bootstrap 
program described in Chapter III, loading in logical 
sequence each module’s kernel. The details of this portion 
of kernel reinitialization are related in Chapter III and 
thus only a brief overview, highlighting the bootstrap 
program’s tasks, will be presented in this section. 

The Load CPU, executing in the global bootstrap 
program, will be tasked to reload each individual modules 
distributed kernel into a global memory buffer. Once this is 


accomplished the Load CPU will determine the particular 
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module designated for the kernel just loaded. Using the 
kernels deSignated module identification (affinity) the 
Load CPU will signal tne target processor desired, by 
filling in the target CPU’S mailbox in the Configuration 
Table with the address of the bootstrap program. After the 
target processor detects that it’s mailbox has been filled, 
it will exit it’s wait state in the error rovtine program 
and transfer control to the bootstrap program. The tareet 
CPY will then proceed to reload it’s kernel file from the 
global buffer into it’s own local memory with the result 
Demme a r@€initialized kernel. The targé@t processor then 
Signals the Load CPU, via it’s acknowledge entry in the 
memereuration Table, that it has completec reinitializing 
it’s own kernel. The Load CPU will then reload the next 
meme from s@€condary storage in the same fashion. This 
sequence of events is continued, under control of the Load 
Mate until all system modules have nad their respective 
kernels reinitialized. 

Upon completion of the kernel reinitialization 
routine the Load CPU will signal all processor modules by 
Setting it’s own acknowledge flag in the Configuration 
moore. This will force all processors to execute an indirect 
procedure activation (a call by pointer) to transfer control 
from the bootstrap program to each modules respective 
Kernel. This jump to the kernel will te accomplished in the 


Same fashion as outlined in Chapter III, only the parameters 
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passed to tne kernel in this instance will be of a different 
Meraety. In additon to the logical CFU identification of 
Pech particular processor performing the control transfer, 
the arguments will include the location of the Configuration 
Table (a pointer) and it’s global bus lock variable. Note 
the uniaue physical processor serial number iS not required 
to be passed as a parameter as it is contained in the 
Configuration Table. 
o. Configuration Table Reinitialization 

During the reloading of the distributed kernel 
each individual CPU has the responsibility of reinitializing 
the Configuration Table to reflect the memory pages 
allocated to it’s own kernel. Additionally, the Load CPU is 
tasked with reinitializing the global memory map to identify 
the memory reserved for the Configuration Table and the 
global bus lock variable used to control access to the 
Configuration Table. 

PEmice sp neeUOOtTSUnaD Droeram Executes on The bare 
system hardware (viz., with no operating system Support), as 
did the bootstrap program of Chapter [II, the memory 
allocation mechanism of the kernel iS not available to 
distribute and record memory usage. This does not present a 
difficult memory mapping problem, during reinitialization of 
the kernel, as the programs and data structures loaded by 
the bootstrap program can all have constant locations in 


memory. The ability to locate these programs and data 
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structures at absolute addresses is realized by the fact 
that these processes will be the first reimeriaiized 
programs. This means that all the old system code can be 
over-written. 

Each module is responsible for recording, in the 
Configuration Table, the local memory pages allocated for 
meee Kernel it reloads. Since the location and size of the 
kernel are known, after an individuel module has reloaded 
it’s kernel, it is a simple matter to reinitialize the 
Configuration Table to reflect the memory pages in which the 
kernel resides. 

The Load CPU is responsible for reinitializineg 
the global memory map to reflect the memory allocated to the 
Configuration Table and it’s global bus lock variable. This 
memeon is accomplished as tne first set of instructions tne 
Load CPU executes in the bootstrap program. The Baeta CPU 
first indexes through the global memory map setting the page 
entries for the Configuration Table and it’s bus lock 
variable to unavailable and all the other page entries to 
free. Note that the convention used to indicate a free page 
im, the bit map is 4 one, while zero indicates a page has 
been allocated. This allows an all zero setting to indicate 
a full memory map while non-zero entries indicate remainine 
free pages are available for allocation. 

Zone rne) Interface 


The transfer of control from the bootstrap program 
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to the kernel, of all System processors available to the 
system (i.e., not eliminated by the error routine), will 
proceed in the same fashion as described in Chapter III. The 
meaquence of events executed to interface from the bootstrap 
program to the kernel will be presented in this section, but 
tne detailed mechanism involved will be left for the reader 
to review in chapter III. 

necall that the transfer of ‘control to the kernel 
is executed by all processors after reloading of the kernel 
(by all modules) iS complete. This procedure was required to 
allow the kernel to commence execution properly with all 
Kernel processes and Synchronization structures established 
maa consistant state. 

Once the Load CPU has signalled all CPUs to 
memmorel to tneir respective kernels the reinitialization of 
the distributed kernel can be considered complete. The next 
memence of events will Entail the réinitielization of the 
application processes. In order to support the relocation 
meune that will be employed to reload the application 
proceses the address of the Configuration Table and it’s 
Controlling global bus lock variatle must be passed to tne 
kernel. Additionally, the logical CPU identification of eacn 
processor must be passed to tne kernel during individual CPU 
e@ontrol transfers. This will ensure the logical 
identification of each module in the system and facilitate 


individual processor memory map location during tne dynamic 
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relocation process. 

The parameters mentioned above are passed to the 
mermel On tne stack of the bootstrap program. The kernel 
interface sequence of instructions will be required to 
remove the parameters passed to the kernel on the stack and 
designate locations in the Processor Data Segment (FRDS) 
(17] for these structures. Additionally the kernel interface 
sequence will be required to establish the correct kernel 
Smerronment for execution by transferring control to 4 
special reinitialization program that will reinitialize the 
Semeeeestructures used by the xernel. Recall that the kernel 
Paterface sequence of instructions occur in the eats 
Scneduler of the operating system [23]. The readers 
attention is directed to the detailed description of the 
kernel interface initialization Sequence in Chapter II!2. 
This procedure performs the identical function as the kernel 
interface initialization Sequence used during automatic 


system recovery.. 


D. APPLICATION PROCESS REINITIALIZATION 

The reinitialization of tne users application processes 
employs a kernel loader process. It is the responsibility of 
the kernel loader process to reload the applicat? on 
processes once the distributed kernel has been reinitialized 
and has restarted execution. Essentially the kernel loader 


process performs a reinitialization of the application 
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processes, eStablishing a known correct state (that of the 
original initialized system) from which the system can 
restart execution of Pe asmuoriicale tas kor. 

Reinitialization of the user’s application processes 
begins with each physical processor commencing execution in 
it’s own kernel loader process. The sequence of instructions 
executed, once the Kernel initialization has been completed, 
to allow the kernel loader process to run are summarized by 
Wasson [23]. Essentially they entail reinitializing the 
Virtual Processor Map [23] of every kernel to reflect the 
loader process as the highest priority process ready to run 
on any processor. This has been accomplished by the 
reinitialization of the kernel data structures Cs iene 
reloading. This e@nsures that all processors will load and 
run their loader processes first once kernel execution 
commences. 

The reinitialization of application processes involves 
reloading the application programs using a new system 
configuration in which faulty modules have teen eliminated. 
Since faulty components are eliminated on the module level 
of processor and memory (i.e., an iSBC 86/124) those 
application processes assigned to a faulty module are 
reassigned, during reinitialization, to a module that is 
mumgectioning properly. 

The ability to reassign the application processes during 


reinitialization to different modules (once a module is 


oa 





eliminated) is based on the use of identical modules. Since 
all processor and local memory units are the same (i.e., all 
aTe iSEC 86/12As) the application processes are capable of 
executing on any module. Note that specific applications 
programs may impose restrictions that will not allow 
reassignment to just any available module. These 
mestrictions might be due to the leneth of a program (i.e., 
it is too large to be reassigned to a module that already 
has processes assigned). In this case a Spare module might | 
be assigned if available. The specific restrictions imposed 
by an application process concerning its reassignment will 
be discussed later in the chapter. 
1. Segmentation 

The apittty Of the Frelinitilizgation Proutine to 
reassign the application processes to different modules 
during automatic system recovery is dependent on the use of 
segmented memory. Segmentation allows each application 
process to have a defined address space that can be 
specified by a distinct group of segments in memory. Shared 
seements cHimmeni st liv the ea@aqdress  “Sspdce For, 9 mul tiple 
processes for the purpose of inter-process communication, 
while individual processes can be isolated from otner 
processes by using unique segments that are not shared. 

Segmentation of memory iS Supported by the Intel 
hardware associated with iSEBC &6/12A module. Recall that the 


one megabyte of addressable memory available to tne &¢&E CPU 
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provides segments up to 64 kilobytes long [5]. Although 
explicit segment boundaries are not enforced, the use of a 
segment manager to allocate memory, based on a predetermined 
page size and segment length, will allow the manipulation of 
Gumemrocesse€s address space. This, in turn, will support 
dynamic relocation. 
2. Dynamic Relocation 
Reassigning the application processes, during 
Mmeeeetialization, is made highly flexible if the ability 
exists to relocate the segmented address space of the 
processes. The capability to relocate the application 
processes facilitates reloading these processes at different 
locations in a newly assigned module’s local memory or in 
global memory, thus utilizing available memory effectively. 
The automatic relocation of the application processes, 
Morange ireinitialization procedure, is known as dynamic 
relocation. 
a. The Compact Compiler Option 
Dynamic relocation is made possible if no 
absolute memory addresses are contained in 4 processes 
address space. The ability to dynamically relocate the 
application processes, during reinitialization, is 
facilitated by using the compact option of the PL/M-86 
compiler [7]. All code compiled using the compact compiler 
Bpelon is placed in elther @ code, data, stack or optional 


user defined memory segment depending on its use. Fecause 





only these four segments are allowed (i.e., all code is 
compacted into one of the four segments) the segments remain 
unchanged during the lifetime of program execution. This 
means that the Code Segment (CS), Data Segment (DS), and 
Stack Segment (SS) registers of the 8@86 CFU are fixed and 
thus not changed during program execution. Consequently all 
meoeswreterences are reflected as offsets from the CS, DS, or 
SS registers and no absolute addresses are entered ina 
processes address space. The placement of offsets in the 
Geyect code, by the utility locator routine (LOC&6) at 
mmemmemeecene@ration time, facilitates relocation of a process 
during reinitialization in that the absolute address of all 
segments of process can be changed by altering the &¢8&6 CS, 
DS, or SS registers. 
bd. The Prologue 

Avot Totel object "Files, created using the 
PL/M-86 utility routines [6], invoke a program prologue at 
the start of execution. This prologue is designed to 
establish the address Space of the program to be executed by 
setting the appropriate fYregisters in the 8@86 CPU. The 
prologue will differ depending on how the program was 
compiled. For the automatic system recovery design, the 
compact compiler option was employed as it provided the mest 
flexible environmert for cynamic relocation. 

Since all code compiled with the comvact option 


exists in one of four segments [7], the 8@86 CPU’s cS, DS, 
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and SS registers are required to be set only once as they 
remain unchanged during program execution. The program 
prologue of a compact Compiled program will set tne CS, DS, 
Good registers prior to program execution. In order to 
relocate the application processes, compiled using the 
compact option, the program prologue for a process must be 
avoided so that the 8@86 CPU registers can be set to reflect 
a possible new process location after reinitialization. This 
can be accomplished by creating, essentially, a new program 
prologue (in the form of an assembly language program, as 
Shown in Figure IV-4) tnat will not set any of the &@@8E CPU 
registers. The function of this ‘Start program for each 
application process will be simply to perform a short jump 
to the start of the actual entry point address of the 
Soper ication process. This allows the 8@86 CFU registers that 
define the address Space of a vrocess, during execution, to 
be set to reflect a possible new location of the application 
process. 

The simple start assembly language program will 
allow the normal program prologue of the application 
programs to be by-passed (i.e., no CPU registers are set). 
As Figure IV-4 shows this is accomplished using just the 
offset of the start address of the application program. This 
moore jump to the application program entry point, using 
only the address offset, facilitates program relocation by 


Allowing the code to be independent of absolute addresses. 
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» START .ASM 
» INITIALIZE THE APPLICATION START ADDRESS 
3 AS A DOUBLE WORD VARIABLE 
START-DATA SEGMENT 

AE PL=S TART ADDR DD G29S:O9006 
START-DATA ENDS 
START SEGMENT 

ASSUME CS :NOTHING 

ASSUME DS :NOTHING 


ASSUME SS :NOTHING 
ASSUME ES: NOTHING 


» MOVE THE APPLICATION START ADDRESS 
> INTO THE AX REGISTER AND DO A SHORT JUMP 
MOV AX, OFFSET APPL-START-ADDR 
Wee AX 
START ENDS 


END 


START ASSEMBLY LANGUAGE PROGRAM 


Figure IV-4 
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c. The Process Definition Tatle 

The manipulation and relocation of a4 process’ 
segmented address space, during reinitialization of the 
application programs, is primarily supported by a global 
data structure called the Process Definition Table (PDT), as 
defined by Ross [28]. This structure is created by the 
System programmer at system generation time and identifies 
Meemeacaress space of every application process that will be 
loaded (or reloaded) to run on the system. Since the address 
Space of every application process is known, prior to 
commencing system execution (viz., all segment sizes have 
been established for the run-time, static environment), the 
PDT entries can be predetermined at system generation time. 

The primary function of the PIT is to associate 
a group of segments with each application process, thus 
establishing a unique address Space for each application 
process. The PDT is reloaded into global memory at the sare 
time that the reloading of the kernel iS accomplished. The 
kernel loader process then uses the PDT to recreate the 
application processes aS reinitialization is performed. 

The PDT, as Shown in Figure IV-5, is a static 
Structure, the size of which is predetermined at system 
generation time asa function of the number of application 
process to be used in the system. The PDT is indexed by 
meercel process number which will identify the processes to 


the system reinitialization mechanism. The first entry in 
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meemwrDT, called Processor Configuration Mapping (PCM), isc an 
array that determines the configuration of the system. This 
array serves Dome aSSOciate, Or Map. =Specittic logical 
processors to individual application processes and is 
indexed, in decreasing order, by tne number of modules (iSRC 
86/12As}) available to the system during the reinitialization 
moutine. Tae Processor Cont teuretionewsdprang | entries 
establish a processor affinity, for a particular application 
process, as a function of tne total processor modules 
remaining in the system during automatic SyStem recovery. 

The ability to dynamically reconfigure tne 
System uSing the logical CPU affinities designated in the 
Processor Configuration Mapping is based on the use of 
identical modules (viz., the unique physical identification 
of a module is not necessary). For example consider a system 
which originally consists of eight modules (i.e., eight iSEC 
86/12As). The modules are simply assigned to application 
processes by a logical number between zero and seven in the 
PCM entry that reflects eight modules are available for 
system use. Once a module fails, the remaining seven modules 
are reassigned application processes tased on the logical 
mm~mrres in the PCM and the predetermined configuration for 
seven available processors in the system. 

Tne processor affinities for a Peet cu ar 
application process are established at system generation 


time by the system programmer and must be Ca Gea y 
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coordinated to ensure continued system operation as the 
processors are diminished. Note that a minimum number of 
processors is usually required to sustain correct system 
operation and this number is reflected by the last entry of 
the Processor Configuration Mapping (PCM) array. 

Ago plonaliy the PDT willl icontaim an entry ror 
the process priority (PRIORITY). This will be used by the 
kernel to establish a preempt priority during system 
execution. Following tnis will te a process register entry 
(PROCSREG) that can be used to establish any 8086 CPU 
register settings (otner than the segment registers) during 
mes reinitialization of the application processes. In most 
Bases only the Instruction Pointer (IP) will be set and all 
the other register values will be reinitialized to a null or 
zero setting. 

Ae last ertries in the PDT establish an 
individual application process” unique address space (PAS). 
These entries will consist of an array in which the first 
three entries will be dedicated to the Code Segment (CS), 
Data Segment (DS) and Stack Segment (SS), respectively, of 
momapplication process. The remaining entries will be used, 
as Yrequired, to provide the identification of any external 
Shared Segments that exist in a particular application 
process’ address space. The maximum number of external 
Segmerts are fixed at system generation time and are a 


Eunction of rae application processes and their 
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mequirements. The entries in the address space array of the 
POT will be unique logical numbers that will identify 
individual segments in another global data structure, used 
during reinitialization, Called the Global Active Seement 
Table (GAST). This structure will be described in the next 
section. 

Vice last. field of the Process, Definition sable 
(PDT) is a bit map identifying an individual segment’s 
attributes. In particular thie bit map uses a zero (8) to 
Sienify if a Segment is only readable (R) and a one (1) to 
mark a s@gment as readable and writable (R/W). A segment 
attribute will be required by the Segment manager in the 
Mmemoee to adetermine whether a segment is to be relocated in 
global or local memory during reinitialization. 

ad. The Global Active Segment Table 

The Global Active Segment Table (GAST) is a 
meopal data base structure employed by the kernel loeder 
process to reinitialize the application precesses. at 
merrorms essentially the same function as the GAST descrited 
by Moore and Gary [14] in their memory manager design; it 
provides a listing of each individual active segment used in 
the system (for the run-time, static system design all 
Segments are considered to be active). The GAST identifies 
the auxiliary storage address of every segment used by the 
system application processes and associates a logical 


mamper, corresponding to tne GAST index, with every séement 
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established in memory by the systems programmer. 

The GAST, as Shown in Figure IV-6, is created, 
as was the PDT, at system generation time and reload with 
the Kernel. The size of the GAST iS determined by the 
maximum number of apdlication processes in the system and 
the maximum number of authorized segmentS per process 
address space. 

The CAST is indexed by Segment number. The 
logical index of eacn segment in the GAST will be entered in 
the PDT at system generation time to allow each Segment in 
an application process” address space to te identified. This 
convention will provide the Segment manager process, in the 
kernel loader, with the ability to access each individval 
Segment in the system for reloadine during process 
mmepidiizgation. 

The secondary disc address of a segment will be 
contained in the first field of tne GAST (DISCSADDR). This 
absolute disc address will be used by the kernel loader 
process to reload the segment during arplication process 
reinitialization. A null entry for tne disk address 
indicates that the segment (e.g2., a data buffer) must be 
allocated main storage, but nas undefined initial contents. 
The Global Address field (GLOBALSADDR) of the GAST will te 
used to indicate if a segment resides in global memorv. If 


the global address field is set then the seegrent is located 
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THE GLOBAL ACTIVE SEGMENT TABLE 


Figure IV-6 
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in global memory. If the field is rull then the segment must 
be located in local memory. 

The CPY Local Active Segment Table kntry 
(CPU-LASTE) is uSed aS a connected processor list. The field 
igs an array structure whicn is as large as the maximum 
number of processors originally allocated for the syster. 
The entries in this field provide an index into each 
processor’ s Local Active Segment Table (LAST) and will te 
used by the segment manager in the kernel loader process to 
manipulate Segments during process reinitialization. The 
length of segment is ccntained in the Size Field (SIZE) of 
the GAST. This entry is used by the segment manager process 
Geeeetne kernel loader to allocate the appropriate amount of 
memory for the segment during the reloading of application 
meocess réeinitializatrion. 

e. The Local Active Segment Table 

Tne Local Active Segment Table (LAST) is 
employed during reinitialization for the purpose of memory 
allocation in the same fashion that Moore and Gary [14] used 
it in their Memory Management Unit. The LAST (see Figure 
I¥-7) is a processor-local data base in the form of an array 
that records the local memory location of all segments 
memoaged On a particular processor module. The index into 
the LAST is reflected in the GAST’s connected processor list 
(CPU LASTE) for eacn individual segment in the system. The 


LAST entry in the GAST is used ty the kernel segment 
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THE LOCAL ACTIVE SEGMENT TABLE 
Figure I[V-7 
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manager routine to locate segments previously reloaded that 
muSt be moved to global memory due to their being shared ard 
writable. 

5. The Kernel Loader Process 

Reinitialization of the application processes begins 
once all processor modules have entered the kernel Loader 
process (see Appendix D). Recall tnat tne kernel nas been 
reinitialized so that once it starts execution, the Loader 
Mrocess, vd€ing the highest priority process ready to run, 
will be the first kernel process executed. Since the logical 
processor number of every CPU was passed, when control was 
transferred from the bootstrap program to the kernel, all 
moomwles maintain their logical identity. This means that one 
particular CPU still has the title of Load CPU. It is this 
MmeeeessOr unit that will coordinate application process 
reinitialization during automatic system frecovery. 

Pie ernie] Loader process is required to reload tne 
application processes sequentially according to their entry 
in the Process Definition Table. Reloading of the individual 
applications processes one ata tire viz not 
Simultaneously) is mecesSsary primarily due to hardware 
mummveations. In particular, as described in Chapter III, not 
all processors will have access to secondary storage thus 
requiring the Load CPU to perform system 1/0 using a primary 


memory global buffer that the remaining CPUs can access. 
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ae. oe Load ¢ Py 

ire ead weet will execute some Instructuons in 
@aemeerne! Loader process that the other processors will 
not. In particular the Load CPU will have the responsibility 
of sequentially indexing through the Process Definition 
Table (PDT) identifying each application process and the 
physical module into which it will be reloaded. fThe 
association of a processor and an application process to be 
reloaded is accomplished using the Processor Configuration 
Mapping field (PCM) of the PDT. Recall that this mapping is 
based on the number of physical CPUS available to the 
system at the time of relntialization. The mapping 
configuration of the processors includes all combinations of 
processors from the maximum available down to tne minimum 
required to continue correct system execution. Tne Load CFU 
Memeeenot co the actual reloading of the application process, 
but will signal (via the ITC Advance procedure [23]) the 
processor module associatec with the process, in tne PDT, to 
perform the task. Note that although the automatic recovery 
mechanism is based on tne use of identical processor 
modules, future expansion of the design might include 
special processors (i.e.,a Multiply CPU). It would tnen te 
necessary to use the Configuration Table to identify a 
specific physical processor and it’s associated logical 
number. 


The particular processor signalled by the Load 
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CPU is a function of the mapping configuration associated 
with an applications process in the PDT and the number of 
CPUs available to the system during reintialization. Note 
that if the processor reyguired to reload the application 
process is the Load CPU, tne reinitialization of that 
particular process iS performed by the Load CPU. After 
accomplishing the reloading, the Load CPU will just index to 
the next process in the PDT. 

Once tO Came: Ue Comme he ri ne cummnc Cummins 
affinity (the processor associated with a process through 
the configuration mapping) for a particular process, and 
Signalled (via ITC Advance) the target modules loader 
process, the Load CPU will enter a wait State {The 
reintialization of the application processes uses the [TC 
eventcount Synchronization procedures of Advance and Await 
[23] ). The Load CPU will remain in a wait state urtil the 
target processor signals (by an advance on the Load CPU's 
eventcount) it has reloaded, and thus reinitialized, the 
memmened application process. This sequence of events is 
repeated until all applications processes listed in the PIT 
are loaded into tne modules they have been assigned to. 

While the Load CPU is indexing tnrough the PDT, 
mrenalling tne appropriate CPUs when it is their turn to 
reinitialize a particular application process, the remainirg 
processors wet have entered a wéit state in their 


respective kernel loader processes. This synchronization is 
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Similar to that performed in Chapter III, only the more 
flexible kernel eventcount primitives are now available to 
support processor communication. Once a processor, other 
than the Load CPU, has completed tne reinitialization 
process, it will return to a wait state, remaining in that 
State until signalled to reinitialize another application 
process or until system restart is executed. 
bl Swap-in 

The Swap-in procedure is called by the kernel 
loader process to freload, from secondary storage, an 
application process. Swap-in is designed to reload a 
specific segment in the address Space of a process and 
Besurm ithe start address of that relocated segment. Moore 
and Gary [14] originally developed the Swap-in routine for 
their memory management unit and it is a modified version of 
their design that is used in the Kernel Loader Process. 

The ability to incorporate a portion of the 
Memory Management Unit designed by Moore and Gary is the 
result of the fact that tne Memory Management Unit design 
and the Automatic System Recovery mechanism are baSed on the 
same family of distributed operating systems originally 
developed by O°’Connell and Richardson (15]. The hierarchal 
design of the operating system provides a significant 
advantage in that it is relatively hardware independent and 
thus compatibility between systems is feasible. 


When signalled (by an eventcount advance) to 
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mesoed adn application process, tne target CPU will te 
required to sequentially index through the address space of 
that process in the PIT. Swap-in will be repeatedly called, 
by the target processor’s Kernel Loader, to reload each 
individual Segment in the process’ address Space. Each time 
Swap-in is called it is passed the logical segmert number in 
the PAS array of the PDT. Recall] that the logical Segment 
number is used to index into the GAST. Swap-in will be 
required to use the logical Segment number index, in the 
GAST, to determine the segmerts absolute disc address on an 
auxiliary storage device (i.e., a hard disc). 

Once Swap-in has established a secondary storage 
address, it will move the targeted Segmrent intdO primary 
Gemomy. The procedure for determing if local or global 
memory should be allocated is defined by Moore and Gary 
[14]. In particular three conditions can be encountered 
during the invocation of Swap-in. The segment can already be 
located in global memory, the segment can te located in one 
or more local memories or the s@gment may not have been 
previously reloaded during this activation of the automatic 
recovery routine. 

If the Segment has not been previously reloaded 
(i.e.,the GAST Global Address and the CPU LASTE fields are 
null) then the segment iS reloaded in local memory as 
defined by the process affinity and the appropriate entries 


in the GAST’s connected processor list (CPU LASTE) and the 
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LAST are made. If the segment nas been previously reloaded 
into global memory (as evidence of the GAST reflecting a 
global address) then it is not necessary to reload the 
segment. Only the GAST and the LAST need to be updated. 
Finally if tne segment already resides in one or more local 
memories, i1t must be determined if the segment is writable. 
This 1S accomplished using the PDT Read/Write bit map. If 
the segment is writable and located in another modules local 
memory (as reflected by the GAST’S connected processor liSt; 
CPU LASTE) it must be moved to global memory where it can be 
Shared and the global address in the GAST filled in. If the 
segment is only readable then is may be allocated local 
memory and the LAST updated. 

Once the memory space has been allocated for the 
Segment, as determined by the size field in the GAST, 
Swap~in will reload the segment and update the Configuraticn 
Table memory mapSs returning the seement location to the 
Berne) loader process. The loader process will then é@nter 
the segment’s location in the Process Parameter Block (PPB). 
The PP2 is a local data Structure that is used to Tecord all 
the locations of the segments in the process” address space 
Peloaded by Swap-in. 

The sequence of events executed, once Swap-in is 
Galled, will be repeated until the Loader Process has 
indexed completely through the PAS array or until a null 


mmeerye 1S «discovered in the PAS indicating ail the process 
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segments have been reloaded. The Loader Frocess will then 
call CYreate-process, pasSing the locations of the Segments 
just loaded, to complete the reintialization process. 
c. Create-process 

The Kernel Loader process will Sale the 
procedure Create-process to culminate the reinitialization 
of the application processes. The Create-process routine is 
an Operating system (kernel) routine designed by Wasson [23] 
and implemented by Rapanzikos (a7 eee Sent idiom, t 
reinitializes entries in the process” stack Segment that 
define tne process” address space. The process” stack is 
then used by the kernel to establish a Dab teu iar 
application process” run-time environment. 

Create-process will be passed the address of the 
Process Parameter Block (PPB) each time it is activated by a 
particular CPU Loader process. Recall that the FPEB is a 
local data base used to record tne locations of all segments 
in the application process’ address space. The Stack Segmrent 
(SS) for eacn application process will be created using the 
PPB and the PDT processor register array (PROCSREG). Once 
Create-process has reestablisned a process” address space 
and reinitialized the register values on tne application 
process” stack it will place tne process in a wait state. 
All processes are recreated in a wait state by 
Create-process waiting for a system start event (i.e., an 


Advance on the system start eventcount [{17]). Control will 
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Maen return to the kernel Loader process. 


BE. RESTART 

Once the Load CFU has indexed completely through the PDT 
the task of application process reintialization is complete. 
The Load CPU is tnen required to restart the system so that 
normal, mana Tee eTreicution can resume. This aS 
accomplished by the Load CPU performing an Advance [17] on 
the system start eventcount. Recall that all application 
Mummeiooes are recreated by Create-process suspended in a 
wait state waiting for the system start eventcount to te 
Memeced. After this event takes place all processors will 
resume normal operation dy executing the nignest priority 


application process assigned. 


Fs. APPLICATION PROCESS STRUCTURE 

In order to facilitate dynamic relocation during the 
mpomatic system recovery process, some restrictions must te 
imposed on the structure of the applications pregrams. It is 
the purpose of this section to outline these restrictions 
and additionally provide some insight into their requirement 
in order that the applications programmer might better 
perform his programming tasks. 

Fach application process is determined by a segmented 
address space that can be defined by unique code, data, and 
Stack segments (using the compact compiler option [7]). 


Since these segments are unique (viz., not snared) a scheme 
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por segment sherine , to fae ieee Enter process 
communication and Synchronization is required. 

Shared segments are created, at system generation time, 
by adding additional segments to a process’ address Space. 
These external segments are then reflected in tne PDT, 
associated with each Dave ta Cul aire oD eric cut) © Mamma Cie. es 1S 
depending on process communication and Sy¥ynenrons Zat. on 
requirements. The external segments of each process will be 
peioaded during process reinitialization and as a result of 
the procedure Create~-process, their locaticns will be placed 
in the unique stack segment of each individual application 
process. The stack of each process 155 in errect,. a) unique 
Memeranution segment that contains pointers to all sSe€ements 
in a particular application process” address space. Hardware 
segmentation tnen allows the stack segment of an application 
process to be employed as a parameter list of pointers as 
described below. 

Woen system automatic recovery occurs, all application 
processes are recreated by the reintialization routine ard 
thus the external shared segments, as well as the unique 
code, local data and stack segments, are updated to reflect 
any changes in segment location. Tnis results in a newly 
created stack segment that will reflect the reinitialized 


address space of an application process. 
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ioe The Entry Point 

The restriction placed on the structure of an 
application process is directed @ Gee en ony Onin GO hemes cir t 
address of the intial procedure. When the kernel activiates 
a particular application process it will use the stack 
Segment of the process to set the code and data segment 
registers of the &@86 CPU. Since there are not enough 
physical registers to allow all external segments ina 
process to be set, a Scheme must be devised so that the 
process can reference all it’s external segments. 

The convention to do this exploits the entry point 
to the application process. This will take the form of a 
procedure in which the external segment locations will be 
passed aS pointers. Requiring the application process Start 
address to be a procedure entrance will permit the process 
to use the preset external SyStem pointers on the process” 
stack to define the formal procedure parameters of tke 
application program. Note that the Stack pointer (SP) is Set 
(as defined at system generation time) to indicate the first 
external Segment pointer on the Stack. 

The applications programmer need only te concerned 
with parameter ordering in the applications process. The 
burden of parameter organization, in terms of stack 
Structure, rests with the Sytem programmer at system 
generation time. Specifically tne systems programmer is 


required tO make the appropriate entries in the Process 
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Definition Table (PDT) to provide the logical ordering of 
the external pointers in the formal parameter list of the 
application procedure. 
2. External Variables 

The external segment pointers, contained in the 
memicdis parameter list of tne application procedures are 
declared as FL/M-86 pointer variables. The applications 
meoerammer is then required to use these pointer variables 
to reference PL/M-86 based variables [5]. This action will 
result in the process’ external segment base addresses being 
used as pointers for addressing the external shared data 
Structures employed in the application process fon 


inter-—process communication and synchronization. 
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VV. CONCLUSIONS 


A. SUMMARY OF RESULTS 

This tnesis has focused on a technique for automatic 
System recovery designed to provide the fault-tolerant 
MoeratioOn Of a real-time, distributed multiple microcomputer 
system. The initialization mechanism developed by Ross ([(2¢} 
Besos implemented and tested as tne first phase of tne thesis 
effort and proved to be a solid base from which 
reinitialization could be accomplished. To Support the 
reinitialization routine, which emplcyed complete reloading 
of the system processes, a method of dynamic relocation 
exploiting the Intel hardware was developed. This lead to 
the ability of the system to dynamically reconfigure after 
the Climination of a faulty system module. 

The fundamental concepts developed as the result of the 
researcn efforts of this tnesis provide the basis for 
fault-tolerance in a system where temporary data loss is a 
tolerable condition. The ability to completely reinitialize 
the system while eliminating faulty components is 4 
desirable attribute in many real-time systems. The automatic 
system recovery design presented in this tnesis is the basis 
for fault-tolerance in a freal-time system that has a 


multiple microprocessor environment. 
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|B. FOLLOW-ON WORK 

Ans thesis addressed only one aspect of 
fault-tolerance; that of tault recovery. AS the introduction 
revealed, the elements of fauilt-detection and 
fault-diagnosis are uSually included ina fault-tolerant 
computer design. Research concerning fault detection and 
fault diagnosis will provide a challenging area for 
follow-on work. Specifically the error routine discussed in 
Chapter IV must be developed to Support the automatic SyStem 
recovery mechanism. Only with fault detection and diagnosis 
routines incorporated will the antomatic recovery routine 
provide complete Pave’ tolerance tO a loues [uj blis mel oesie: 
microcomputer system. 

Dynamic reconfiguration in the automatic system Bee 
design revolves around the processor/memory module (the iSBC 
86/124). Further research might specifically investigate the 
Separate reinitialization of only faulty memory. The logical 
extension of the recovery mechanism lends itself to the 
possibility of saving the fault-free portions of memory ix 
the form of the PDT and GAST. These data bases would then 
allow the error routine to eliminate Specific sections of 
faulty memory and record the memory removed. This, in turn, 
would allow a reduced reloading requirement and thus a mere 
expeditious execution of the automatic system recovery 
routine. 


The autOmatic recovery design presented by this thesis 
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provides a basis for fault recovery. Furtner development of 
the design could proceed in numerous directions with the 
concepts of dynamic relocation and reconfiguration 
facilitating a variety of specialized designs. For example, 
an eXpansion of the automatic frecovery mechanism might 
include check-pointing, where data processed prior to a 
System failure could be saved; tous reducing the 
reintialization requirements. The automatic recovery 
mecoanism might also te used in conjunction with otner 
mecoevery techniques. In particular reinitialization mignt be 
used in a2 system that employs redundancy. A specific group 
een, cluster) of faulty microcomputers could te 
memo? tialized to Climineate tne faulty module woile a 
parallel cluster is substituted to perform the identical 
computations. 

The automatic system recovery mechanism was developed to 
integrate with a distributed hierarchical operating system. 
The original distributed operating system kernel 
implementation developed by Wasson [22] was not specifically 
designed to incorporate fault-tolerance. Although this 
thesis attempted to provide the interface to the operating 
system the continued development of the kernel will 
necessitate additional follow-on work to ensure a compatible 
integration of the automatic system recovery mecnanism witn 


the kernel. 
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APPENDIX A. SYSTEM INITIALIZATION IMPLEMENTATION 


A. OBJECTIVES 

This appendix is provided to further acquaint the reader 
with tne system initialization mechanism presented in this 
thesis. To demonstrate the initialization capability 
Broviaged by the program listings in Appendiz 8 and C, 4 test 
program was developed to simultate an cperatinge system 
kernel. (The test program was required as the previous 
kernel implementation was not specifically designed to 
interface with the recovery mecnanism). The simulated rernel 
was then loaded by multiple iS8C &6/12A single board 
Bompurers in tne same fashion as descrited in Cnapter III, 


using the same hardware Support outlined in Chapter II. 


B. THE SIMULATED KERNEL 

The Simulated kernel program in Figure A-1l was loaded by 
@all iSBC 86/12As and was used to demonstrate the ability of 
Mme initialization mechanism to transfer control to the 
kernel and then commence system execution. The demonstration 
Called for each iSBC 86/12A to have a CRT connected to it’s 
serial I/0 port. Once all simulated kernels were loaded and 
execution transferred to each particular iSBC &6/12A kernel, 
the simulated kernel caused the logical CPU number and the 


unique physical CPU ID of each processor module (iSBC 
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86/124) to be displayed on their respective CRTs. 


C. DEMONSTRATION ENVIRONMENT 

The demonstration environment for loading the simulated 
kernel included all the hardware Support described in 
Chapter II], but due to limited resources only a maximum of 
three iSBC &6/12As were used instead of the eight planned 
for. This required two bootlcad programs similar to the 
listine in Figure B-2 (only the unique physical IDs will 
differ) and a bootload program (used for the MDS-connected 
iSBC 86/12A and thus the bootload CPU) identical to the 


Mmsotsng in Figure B-1. 


Dee sto TEM ACTIVATION 

For demonstration the bootload programs were placed in 
RAM, as described in Chapter III. To initially load all 
three iSBC 86/12A bdoards with their respective bootload 
programs the iSBC 95S7A-iSBC &€/12A interface and execution 
package was employed. In particular the monitor command LOAD? 
was executed to load an individual bootstrap program into 
the MDS-connected iSBC 86/12A"%s local memory. Once this was 
accomplished the monitor MOVE command was used tO move the 
bootstrap program to tne appropriate iSBC &6/12:. (Note that 
Since the local memory of one iSBC &6/12A cannot te 
addressed by anotner iSBC 86/12A tne equivalent global 
address of a particular iSBC 86/12A local memory was used to 


move the code. Also tne MOVE command does not alter any coce 
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Bomererlect a new location; it only provides an explicit 
transfer of code). Additionally tne monitor MOVE command was 
employed to move the four bytes of the pbootload interrupt 
vector to the designated iSBC &6/12A, again using the global 
address. 

The process of loading an individual bootload program 
and it’s interrupt vector into local memory’ of the 
MDS~connected iSBC &6/12A and then moving that code to the 
identical spot in the targeted iSEC 86/12A (using its global 
memory for that location) was repeated for botn iSEC 
86/12A°s not connected to tne MDS. Finally the bootload 
program for the MDS~connected iSBC 86/12A was loaded and the 
initialization mecnanism was activated, using the simulated 
pootload switch: the INTR button on tne iCS-&@ chassis. Note 
mochoe 2G was necesary to start tne MDS—connected iISBC 86/124 
executing a loop, as the MDS interfered with the 
non-maskable interrupt, but tnmat all otner iSBC 86/1245 
commenced execution of the initialization routine from their 


meaopective monitors. 


bee 





‘XHHSLNO ONY 
‘( (Hdd GNY A)LTIOSV)YVHOSENO TIVO 
$((HIO ONY (P°A)HHS)IIOSV) HVHOSLNO TIVO 
‘(, dTGOAVG849GPOCTO, ) VEVI ALAM (x) IIOSVY TUVIONG 
SGLAT © TUVIOI 
$(q)THNGTIONd *XHHSLNO 
/x GUO FHL OL SUATWON XMH SLNMLNO WUNGTIOUd STHL x/ 


SYVHOSLNO ANG 
SuVHO = (HS8aG0) ind Lno 
‘ONG $0 = (HT@ AONV (HV¥GO)LNdNI)AIIHA OG 
‘aLid UVHO THVIONG 
f(YVHO)THNGTIONd sHVHOSLNO 
/x LUO FHL OL SUMLIVUVHD SLNdMLNO AUNAGTIONd SIHL x/ 


$(, TONUGM GaXGENA, )TVILINI FLAG (x) OSWH TUV TOG 


oC) GI Nd IWOISAHd, )IVILINI ALAM (x) OSWEX TUVTOTT 
{(, = GI Ndd TVOIDOT, )IVILINEI ALAA (x) OSWEM TUVTORG 


SabAG I CHVTOIG 


$Od *dOWSXATHS TH NUT 
/x ZINGOW ATId XTH TANYTH NIDAT »/ 


/ este ste sie ste se ste ste ste se 2"e 68 AON O82 TTIg JUS * TANYA ate ote se te ste te ote ate te ate / 


AQ NAN AN 


AN MWY NON 


eta -t 


at 
OT 
GT 
v1 
et 
Zt 


Tl 
OT 


MN M Hw 


SIMULATED KERNEL LISTING 


JUS°TANYAHS TH? GBWId FAT CHNOANI HATIAWOO 
PHO° TANNA THs NI GgovTd ATACOW LOArao 
GOWXHHTENUTY TINGOW JO NOILVIIMWOO 2° TA 98-W/Td II-SISI 


T aoVd 


Figure A-1 


1235 





SONG 


{((I)OSWTIY)YVHOSLNO ITVD 


 $9T OL OB = 
‘(I1)4VHOSLAO 
‘(H9)UVHOS ENO 


§((1)9SWH)YVvHOS ENO 
‘$I Ob 
‘(AT)UVHOSLNO 
§(49)YVHOSLNO 


‘ HVO, ATIVUTLIT a1 


I od 
TIVO 
TTVO 


TTVO 
Q= 

TIVO 
TTV9O 


‘ Hida, ALTIVUYALILT YO WHVIONG 
SaLAG AGI$SNdISSKHd AUNVIIAA 
SHLAG GI$NddI$sa0T MUVIOI 


S(AISNdAISSAHd‘ AISNdISDOT)TUNAGTIONd *TOVANISNYAYH 


‘CNG 


I od 


1 ste ste ste ake 3% stevie ste ste ate ate ate ake ste ate ae We ste ate ate ate ste she ofc ale we ate oe ale ate ate ote ate a ate ate ate ale ate she ate we ate ate ake ste ate ake ake ate ste ss ale ste te ste ate ate ate te ate se ste ste ate / 


/ x “YOLINOW VeT/98 x/ 
/xe AHL OL SNHALTHY NGHL GNV LHO AHL OL YHAWONN TVIYGS Ndd TVOISKHd x/ 
/* @NOIN AHL ONV UAAWON Ndd TIVOIOOT AHL SLNdLNO LI “TANUAY THE «/ 


/ x OLBNI AYLNA GTHL GLVYTAWIS OL dasn ANIDNOU LSAL V SI SIH xe / 
II III IID II OS SS SS SSS SS SS SS x / 
/ se TOVANISNUGH se / 
/ Ske se te ste He ate ve ste He Se ee ae LONE KE EE He ee He He ae LO aNe ae he He KC He Ke ae COKE Re I Be He he fe SNC Re He ae ake ake ate fe abe fe abe oe ake ae he 5 aft Be fe ae / 
SLIXY ang 
STVNHUALXT GUNGTIONd  LIXG 
/x HOLINOW 


Vet/98 JHSI HHL OL NUNLAY OL ASVHd LNAWdOTT ATT ANV 
ONIDONGIT FHL ONIUNG AASN SI AUNTAIONd V4S6 IASI SIHL x/ 


GOWXHHTANYAY 


LOTLAS 


NNN MOMAN WN OD M4 


NANA 


oe 
ae 
ins 
Be 
6c 


Lc 
92 
Ge 
ve 


Le 
CG 
Td 


Oe 


6t 
BT 


SIMULATED KERNEL LISTING 


YHaTIdwod 98-W/Td 


Figure A-1 (cont'd) 


124 





NOILVIIGNOO 98-W/Td dO ANG 


(S)YouuR WVUDOUd @ 


ast H2TO9 = 
Its HES OO = 
aqgt HOTA = 


aSoe2 HAO 


/x (OWSXRTHSTVNUDS  »/ 
/x MHUNGAHOONd ANT «/ !$ROVANISNUTY ONG 


‘LIX TIVO 
$(a1$Ndd$SAHd)XTHS NO TIVO 
Sang 

$((I)OSWEHX)UVHOSDNO TIVO 
‘41 OL @ = I Od 
$(AT)HYVHOISDNO ITVI 
{(YO) UVHOSENO TIVO 
§(QISNdOSDOT)XAHSLNO TIVO 


COWXTHTANET Y 


QVaaY SANIT 2 


aZIS WOVLS WAWIXVW 
AZ1S WadV ATAVIUVA 
€aZIS VXUV LNVLSNOO 


aZIS Waiuv adoo 


*NOTLVWHOUNT FTNGOW 


ANG T i 


OQ 


av 


tP 
OP 
6% 
Bk 
us 
9f 
Ge 


NNN NPN N 


LOIlTS 


HFTIAWOO 98-W/Td 


SIMULATED KERNEL LISTING 


Figure A-1 (cont'd) 


25 





PEGE TXB. 


BOOTLOAD PROGRAM LISTING 


STYNHWDLXA (TLAT TVLOLSOdO “YALNIOd TIVWSNdO 
‘HLA HOVSNdO ‘ALAC GIsndd) 
TUNLINULS (S8)ATTAVLSAdD TUVIOGG 


‘TVNUDTLXT ALAC WANSNdISDOT 
‘TVNUGLXS GLAM WOOTSTALSAdD AUVIIAA 


/ ste ese se se te ae ste te see fe ae ate ae ate a8 oe a8 te af be ate oe ae oe be he ab ate oe fe ae ae af fe ate af ae ae ake abc ae eae ofc abe ate ee oe Ne 28 ae ae ea Be he ae ate te ake / 
/ SNOILVHVTIOTD VLVd T¥€@01) TVNYG LX te / 


/. deste ste ste s8es%e ote teste ste se ate se ake ate fe ate Ne ate ates" ste ate ale ake aig ste ake 8c aK se ae ake ate fe ate ae ate ate ake ate fe ake ae ake ate ste ale ale ake ak ale ake ae ake fe sfe aft afe ake ate ote ate ate / 


‘a, ATIVUALIT NdISaqVOTLOOT 
ae LITVUaGLIT aqI$ndd$SikHd 
‘Guom dHLSLASLVLS 
‘HTLNIOd HAYSdHHLSLISLS 

‘ADAG G1$ndd$901 AUVTIIG 


| se se Ye Ne ste he te te de Pete Bete HE PEELE HE He Se ate e ote te ate He ake ote he ate ole ae ake ote Me ate ote ofc ate ake ake Se cake ate ae ote ae ote ote ake fe ote Be te ate ate te ote teak / 
/ x SNOILYVYIOGA VLVG T¥IOT ote / 


/. ste ste ste a8 3% ste ste ste ate ste ate ale sie ate ake ake se fe ate ake ie afe sie ate ate ote ote ote ate ate ste ae ale ate afc ake ate ale ale ae ae ate ate ote ate ste ste ste fe a%e fe ste ate ate ate ate ote ate ake ate ste ste ate ate / 


(Od ‘MOWSNdOTASLINI 
/x HINGOW Add TVOTLOOG AZITVIGINI NIDA x/ 


J se ste ste ste steske ste ate ste ste ste G93 LOO 22 aTId JHS* LOOANI ae ste ate ate ae ate ae ae ste te / 


MDS CONNECTED BOOTLOAD PROGRAM 


QOUVT OUS°LOOUNI: TH: GSWTd ek CAMOANI HaATIdWOd 
fEO°LOOANI: TH? NI CHOVTId ATACOW Loarao 
GOWNAOIGLINI AINGOW JO NOILVTIAWOO 2°TA 98-W/Td II-SISI 


GqOWNdOTELINI HATIdWOO 98-W/Td 


Figure B-1 


126 





‘49, KIIVHALIT TION 
‘00, ATTIVUTLIT LkAVAUSLON 


© UO ato Pes bee AQGVGUY 
‘TLAG OVTASdaLSLd 
‘TLAG GISNIOSLIVM AUVIIAG 2 an 
S(CISNdOSLIVA)TaNCTIONd PLIVASNdO T tT 
/ de te Ke oe se Ye ate ake te ate ste ake ale ak ole me ae we a8 ale ofe ake ate ake ste ate ofc ste ate ate obe ae aMe ole ae ake fe ate ote ate ake ate ae ake sie ate alt ate ale afe ak oe ole afe ake ake ake ke Ye ste fe ote te ate / 
/ x "WVHDOUd dVN¥LSLOOD AHL ALNIAXA ANY ANNILNOD te / 
/ xe NVD Odd HVINOILUV¥d SIHDL LVHL INIGVOIGNI WVH9CHd dVubs ate / 
/ x -LOO@ FHL AO SSTUAGVY AHL SANTS Ndd AVOTLOOM AHL TIINA te / 
/ xe *“YOOI NIdS ¥ NI ‘SLIVA OL S,Ndd AVOTLOOG-NON HHL SaSsnvo te/ 
le es ee a x / 
/ x LIVASNdO xe / 


/ ste serie s% stesie 8 sleate ote siete Ye ofc sft oe ote te te oleate sfe ste ale oe ake oft ste ate aha ste ote she ate ate she ake ate ote ole 8k ale se ate se ate ofc ate ale ate te afc ate ake ate fe ake ats ake ie ate ake ate ste / 


7 ste se se se se ste ae ae oe ae ae ate ate ate ae ae ae fe ofe ae ae ae ae be abe ake abe abe abe ae a abe Be ae Be fe oe Be fe 28K Be Be ake ae ae ale af ae afc Be Be alee Be ake he ake Re ok ke beat ate / 
[x SdunddI0dd TVIOT x / 


/ ste ste ste ste se ste sie ste a'e ate ste ate ate ate afc te ate ale afc se ate sfc ste ale fe ste fe fe oe oft oft ate ake ate ste ate afc fe a8e ale ate ats ste ate ate ate ate ate ste ste ate sfc ate abe a8 ote te ale ate ate ate ate ate ate / 


-bIXG ANd 
STVNYUALXA GHYNCHIONd -LIXa 


wa ON 


fqvO1T ang 
§QHOM (HOLIMS‘'SVIG) HUVTOGG 
SUMENIOd (SNLVLS‘SAYLNA‘SAWVNATIA) SUVTORG 
STVNUTLXA (SNLVLS'AMING' HODIMSSVIG‘SAWYNATIA)SHNdTIONd saVOT 


MDS CONNECTED BOOTLOAD PROGRAM 


WA NO OQ 
Wm Om @o Oo S& 


| she he Se Se oe ae 2 sete Se se ie oe oe ae ote akc a fe ae ake 8c abc ae ole fe be af ae ale fe ake ae aloe ae ae fe IC ae ake ae ate ae Be ae ae ake ale ale te ate ate abe fe ake abe ae fe ate ae ate ate / 


/ xe SHUNTHIOUd WHLSAS O/I V4S6 OESI TYNE LXG x / 
Nisgsine st eipenieoe Niassa Wye sillier 
LILA 


GOWNdOTaLINI YATIdWOD 96-W/1Id 


Figure B-1 (cont'd) 


n27 





SING 
(T° HOOTS T2L$NdId) LASHIOT ATIHM Od 


‘GLAT IT AV TIOdd 
$2y DUNduayINI AYNAAHIONL = HYLNISAVOTLOOY 


/ ste she ote se se Xe ste ste ote ate ste oN ake he ke ie ofe ake ake ake ake ate ake He ofe ake ake ste ake oN ale ote ake afc ale ale ake fe te ake ote she ake ale ate ale afk ote ofe sfe of ste she a8 ate se ae ate ale ak ofe ate ste / 


/ “aLVES LIVA VY SUMING dI ASIA x / 
/x  -MHHLO “LI Ob SdWwar AGNV WVHDONd dVUYLSLOO AHL SAVOT ULI we / 
/x NHHL (89 = ATIVOIDOT) Add ,AVOTLOOG, AHL SAWOOTA Ndd SIND xe / 
/ x G1 ‘Ndd SIHD WOd GI IVOISAHd GNV TVOIDOT V ACIAOUd OL a / 
/xHMAQ0NYONI GESSHOOV NAHG SI ATavd$Ndd AHL “TuUNAGTIOUd SIHD OL x / 
/ x dWhe S¥SNVO SISSVHOD 2Tt/98 NO NOLENG , YBNI, INILVAILOV xe / 
| rr eee nr rr errr rrr rrr re x / 
Vn YLNISAVOTLOOR 2 / 


/ she ote she ake oe a8 she ste ste ste ate ake ofe ate ate oe s'e ate ate se ake afc afc ole ate afe ate fe ake fe ate oft ofc ake ae ake ake ake ake ate ake ote ofe oft ate ake fe ste se ote ae she fe ate ae sYe ate ae ate ate ak ate ote ate / 


SLIVASNdO ANG 
/xe MUNGAIOUd LIVASNdD ANG x/ ‘NUNLAY 
/x AITHM O10 x/ faNaq 
$9 = YIOTSTALSNdd 
‘JdVaH = YVIASMULSLA 
/x WOOT NIdS LIXGT ANV 
AGVGN OAS NAHL XO@ TIVW NI SI SSHHCIY dVHLSLOOG AI »x/ 
NIHD TION <> TIVWSNdI’ (CISNdISDOT)AIGVLSNdO AI 
/x NGO AQVOTLOOT AM SSHAGAV WVHDONd dVULSLOOG 
OL TTNN WONT LIS NAGA SVH XOM TIVW Add AI ANS OU WOTHO a / 
SaNq 
{( T° HOOTSTGLSNdID) LASHIOT ATIHM OG 
SkAVEH <> OVTISdHYLSLA ATIIHA OF 
‘KAVGUSLON = OVIGSdHbSLE 
/x NIMS OL WOOT AZIIVIGINI x/ 
LOG’ aS 


“%) ee) fs) C0 NN? 


NN i 


9¢ 


0c 


OG 
1A 
Oc 
6l 
ST 


LT 


OT 
GT 
vT 
eT 


MDS CONNECTED BOOTLOAD PROGRAM 


COWNdITALINI YUATIdWCO 98-W/Td 


Figure B-1 (cont'd) 


128 





/x ASTH xx / fang 
STIVNSNdO’ (AISNdISDOT)TIGVESNdD = HAVSdHLSLASLS 
/x WYHDOUT dVULSLOOT AHL OL dwar Od 
GWIL ONITIVNSIS ‘Ndd CVYOTLOOG AHL WONT XOMIIVW Ndd 
QGHL Ob GUSSVd SI LVHL ‘SSHMTGVY WYUDONd dVHLSLOOM LAS x/ 
$(dISNdISDOT) LIVASNdD TIVO 
/% WOOT NIdS ONTLNOWXA AA ALVLS LIV HOGNA x/ 
‘od 
TS 
/x (@ <> A1$NdI$90T1) NdD AVOTLOOR LON AI x/ 


_ /x AI «/ Ona 
‘( dULSLISLVLSO' UAVSdULSLASLSO‘'a‘a'(, dubsdad: ti:,)d)aVOI TIVO 
/x KMOWHW TV€O019 ODNI WVYYDONd dVALSLOOM AVOT x«/ 
SING 
$(GOT)AWIL TIVO 
COO eto Ul 
/x SHATAISATHL JTILNGAGCI GNV AIavisndo 
SSHOOV OL AWIL S$,Ndd UAHLO MOTTIV OL AVITAC GWIL GLVANO »/ 
‘od 
NUHL 4 = AI$Ndds9oT AMI 
/x 0149 AVOTLOOG AHL SI Ndd AI x/ 
‘9 = WOOTSTALSndd 
(T + WANSNdISDOT = WANSNdISDOT 
/x dO DXAN GHL YOM YAAWAN NdD AHL LNAWAUONIT »/ 
(WANSOAdISDOT = AISNddSDOT! 
/x QI NdD TVOIDOT LAS x/ 
‘T + WANSNdISDOT = TVLOLSNdO* (NdISAVOTLOO )XTAVESNdI 
/x GNNOD Ndd CVYOTLOOM LNAWAUONI x/ 
‘dISNdOSSkHd = AISNdI* (WANSNdISDOT)ATAVISAdO 
/* ATAVESNID NI GI Ndd TYOISAHd GNOINA LAS x«/ 


qOWNdOTALINI 


CG 6& 


3e 
4c 


9% 


MH <i ce 


Ves 
mie 
Ck 


Te 
Os 


6¢ 


MDS CONNECTED BOOTLOAD PROGRAM 


82d 


OQ OQ SY} MN AN MN OQ 


Oc 


LOTeLAS 


HATIGWOO 98-W/Td 


Figure B-1 (cont'd) 


129 





NOILVIIdWOO 98-W/Td FO ANG 


(S)YOuNT WVHDOUd B 
QVGX SANIT 2ct 


G2s HVZGO = 
6 H6680 = 
Te HAG99 = 


diay H26t@ 


>NOILVWHOUNT ATAGOW 


AZIS WOVLS WAWIXVW 
aZ1S VAY ATAVIUVA 
aZIS WHuv¥ LNVLSNOO 

aZIS Vad adoo 


/x GOWSNADTASLINI «x/ Sang it uF 
Sana c 8 69F 
‘TO ATIHA OG T GP 
/x “SQW GHL OL GALIANNOD Ndd AVOTLOOM AHL 
NI AINO dOOT NOILNOGXT ALINIANI SALVINI -— WVHDOUd NIVW x/ 
/x TUNGTIONd LANHUALNI ANA «/ ‘HLNISGVOTLOOG ANG 2 F 
$(WIOTSTAL$ Nddd' ATAVESNdI ID CISNdIISDOT)UAV$SdUNdSLASLS TIVO c 06&F 


/x LYOWAW TV€OTD NI WVHDOUd dVULSLOOT OL dWAL x/ 


LOIGlas 


GOWNdVTIGLINI 


UFTIdWOD 98-W/Td 


MDS CONNECTED BOOTLOAD PROGRAM 


Figure B-1 (cont'd) 


130 





STVNUGLXA (ALAT TVLOLSNdD “YALNIOd TI¥WSNd9 
‘EaLAT WOVSNdD “ALAT ail$ndd) 
AUNLONULS (8)ALAVLSndd FAVIOdd 


STVNUATLXT GWLAG WANSNdIS9OT 
‘TYNUTDXT ALAA WOOTSTALSNdD TUVIOTG 


/ ste teste se sterte se sleste se oke ste ate ete ate ste ole oie ate ate ofc ste ake ate ake ae ake ate ate ate ate fe ale ate ste ste ate ale ate ate ote ofe abe ate ale fe ate ste steste ate ste ate ate ste ote ate ste ate ste fe ste ate / 


/ x SNOIDVAVTOTG VLVG TV@OTD TYNUG LX tc / 


| ste Se she she ake oe ake ake ake ake ake se ake ake ake ake ae ae ate ofc ake ae oft ofc ate oe ake ate afe ate ate ate te ate ake ate ale te ale ale ate ste ake ake fe ate afe ste ate ate ate ake ate ate ale fe ate ate ate ate ate ate ate ate / 


sane ATIVUALIT aq1$NddSSiHhd 
‘dUOM d¥LSLESLVLS 
‘YMLNIOd UCVSd¥LSLASLS 

‘ADAT AI$SNdISDOT AVION 


yi se o'e she ote o'e ate ste of o'e a'e a%e ote afe ale ale ote oft ste ate ole ste ate ale ote ale ote ate of ste ote ote abe ate afe ake ste ote o%e ote ale ale ak ale abe fe ofc ofe ofe ats ste sie He ate ae ale ate ate ate ste ake ate ste ate ate / 


ae SNOILHYVIONG VLVA TVIOT te / 
| Se RAC IOR BOR ORI SOI BOF fee ae Sede oe fe de ae ate ale ae afe ae 8k afk ae ate ak fk afk aie ae ote afk ae atk ate ale aie ae ale ae ale afe fe ae afe ae atk ale ae ate ate ate / 


‘Od <dOWSTNddSLINI 
/x QZINGOW TNdD AZIIVIGINI NIDA «/ 


/. Xe ste she ote se ake ote He ie ae ae 338 bLO0 22 atid JHS* TNdAONI ste ae afe te ake ae afc ae ate ate / 


NON-MDS CONNECTED BOOTLOAD PROGRAM 


QDUVT OUS°TNdMONI? Td? GBWId AL CHMOANI ATTIdMWOO 
CTO°TNGINIS TH? NI daOVId ATNGOW Loardo 
QOWTNdUOLINI ATNGOW AO NOTLVYTIdMWOO 2° TA 98-W/Td II-SISI 


QOWTNdOLINI YAFTIdWOd 98-W/Id 


Figure B-2 


Gop! 





‘an, ATIVUILIT TTON 
‘ 93, ATIVUTLIT AQVAUSLON 
,U8, KITVUGLIT hava 
‘TLAG OVIASMULS LE 


‘TLAG GISNdOSLIVA TUVIOA a ZT 
f(CdISNMOSLIVM) AUNGTIONd Ss LIVMSNdd T we 

| ste ste sie he se ae ake ae ae ate ake ate fe oe ake ae ae ake ake ae ake ake fg ae ate le ate ake akc ae 216 af Se ae fe ae eats ee ate fe ae ake aBe fe she fe ate ate fe ate ake aNe ate akc ake ate fe ate ake ste ate / 

/ x “WVH9OUd d¥YHLSLOOD AHL GTHNIAXT ANV ANNILNOO x / 

/ xe N¥VO dd UVINIIDUVd SIHL LVHL ONILVOIQNI WVHDONd dUVELS ste / 

/ x -LOOd@ FHL JO Ssquday THL SANDS Odd AVOILOOG AHL TINO ste / 

/ xe ‘JOOT NIdS ¥V NI ‘GIVE OL S,Ndd JVOTHOOT-NON AHL SaSnvo - 

J xe a ae ee Se eS A Se a ee Se SS Ae SS ee ee ee ae ee ee ee ee ee ee ee eS ea ee eee ——— ye 

/ x LIVASNdO se / 


| ste te se Se ae ate Ke ok a8 a8 ake ae 8 a8 28 ae ake fe ate ae 88 aR Ne ea ae 8 a8 a8 ae ae Bee aR ae Be aR a ae Be Be ae Be a ae kc Be Se ae Fe Be fe fe ae ake te ate ate ate ate / 


/ ste ste ste ste ste ste ste steste ste ste ate se ote ate ae ate te ate oe ate sfe ate ate af ste a%e ate ale ate ate ate ate te s8e ate ate ate ste ste ake ake ale ate ate te ste ate ate ate te ste ste ate ate ate fe ste ate ate ate ate ate ate // 


/* SGUNAGTIOUd TVIOT 


te / 


| ie Se ste se sesh oe Se ke Be te ke Stet I RR ke BOER Ae He Fe ae ee Be ee ae te Fe 8c ae Be eae ae fe ae fe abe fe oe ale fe ake fe alee te oft fe ake ate ae ae Be fe ae ate ate a / 


bIXG 


STVNUTLXT TUNCHOOUd *LIXT 


‘avot 


‘qHOM (HOLIMS'SVIG) HUVIOIG 
SUTENIOd (SNIVLSS AMLNG‘SAWVNGTIA) THVTIOIG 
STVNUGLXA (SHLVLSSXIUINTS HOLIMS‘SSVIT‘SAWVNATIZ) GYNdTIONd :avoT 


ang 


aN 


and 


AON N 
MOM ® O& 


] she de se she eke she se te Be Sete Se ste ote se ae ke Ete Le ate fe He HE Cae a8 HOKE MEK ake she ake aie oe a8 ate ale fe fe ate ate ae ate ate afe ate ate ake ate ae ate ate ate ate ate ake ake ate te ate / 


/ x SHUNGTIOUd WHLSAS O/I VW4S6 JASI TVNYGLXG 


at / 


/ ste ste ste ste ste ste ste sit s'e sig att ste ste ae ote ate ate fe fe fe te ofe ate ate ate fe ate ate ate ale ste ate sfe afe fe aft ate ate 8k fe ate fe ofe ate ake fe ale ale ate ste ate te te she ate ote ee ake ake ote ate ate ate / 


GOWTNdAOLINI 


LOTLAS 


HYATIdWOO 98-W/Td 


NON-MDS CONNECTED BOOTLOAD PROGRAM 


Figure B-2 (cont'd) 


132 





- (NG 
:@OT OL T = I Od 


/x dd AVOTLOOT OL 
LTAVATG OL Add GHLOANNOD SAW MOTTV OL AVTGG GLVauO »/ 


‘qLAd I SHVIORN 
‘cA LdNYYALNI QUNGAIONd sULNISAVOTLOOT 


/, sie ste ste sie aie att ate ake 3% ate aie ate ste ate a%e oe ate ate ate att ae ole ake se a%e ote ate ote ote ake ote ste ote ate ate ake ste ste ate ale ate ate ste ate ate ste ate ate ate ote ote ste ofe ate ate ots ate a8 ate ote ote ate ate ate / 


/ x "aLVLS LIVYM V SUTLNGA FTI ASIAN x / 
/ x -YGHLO “LI OL SdNAL ONY WVUHDONd dVHLSLOOY AHL SAVOT LI ate / 
/x NGHL (8 = AITIVOIOOT) ndd ,AVOTLOOL, AHL SAWOOGT Add SIHD oe / 
/ x GI ‘Add SIHD HOT AI IVOISAHd ANY TVOIDOT VY adIaAoudd Od x / 
/x4ACUONT GQXSSHOOV NHL SI AIGVL$SNd) AHL “TNNACHIONd SIHL OL ax / 
/x* dwohr SHSNVO SISSVHO 2t/93 NO NOLLNAG _HYENI, ONIDVAILOV te / 
| ke rn rn rr rr rn rr errr ———»% / 
/ x ULNISCVOTLOOE ste / 


/ ste steve ste se te se teste ste ake ie she ae ate ate ote te ote ofete ate ake ate ote se ate ate ate ate ale ate ate ate ate te ate ate fe ste ate ae ake ate ate fe ale ake ate ate fe ate ake ake ote ate ate ate ote ate ae ste ate ate / 


SLIVASNdO CNG 
/x% AMNAGHIOUd DIVASNdD ANA x/ ‘NUNLTY 
/x ATIHM O12 »/ SING 
$9 = WOOTSTIL$sndd 
SkAVGU = DVIGSdHLSII 
/x WOOT NIAMS LIXG ANV 
AQVaGH LAS NYHL XO TIVW NTI SI SSHUCAV dVULSLOOG AI x/ 
NUHL TION <> TIVWSNdO’(AISAdISDOT)ATAVESNd) AI 
/x NdO AVOTLOOU AT SSAHACAV WVUDOUd dVULSLOOG 
OL TITAN WONT LAS NAMA SVH XOH TIVW dd GTI HS OL WOAHO x/ 
Sana 
{(T*WOOTSTELSNdIO) LAISHOOT ATIHA OG 
S30 VGN <> OVIGSdULSLA ATIHA OG 
‘2QVAUSLON = OVIASKNLSLA 
/x NIMS OL HOOT AZITIVILINI x/ 
LOTLAS 


QOWTNdOLINI HATIdWOOD 98-W/ Tq 


c mnmnN a 


NON oD Ht 


9¢ 
Ge 


aA 
td 


6T 
3T 


at 


QT 
GT 
oI 
ET 


NON-MDS CONNECTED BOOTLOAD PROGRAM 


Figure B-2 (cont'd) 


133 





/x ASTI x/ ‘ONG 
STIVWSNdI’ (CISNIISVOT)AIAVLSNdD = HAVSdMULSLASLS 
/x WYUNONd dVaLSLOOG AHL OL dwar ou 
QNWIL ONTTIVNDIS ‘dd GVOTLOOM AHL WONT XOMTIVW Ndd 
GqHL OL GUSSVd SI LVHL ‘SSTNACAVY WYHDOUd dVYLSLOOT LYS »/ 
$(dq1SNd0SDOT) LIVASNdD TIVO 
/x WOOT NIMS ONILNOTKT AT ALVLS LIVA MaING / 
‘od 
qSTH 
/x (@ <> GI$NdI$DOT) NAD AVOTLOOT LON AI x/ 


/ se dl se / fang 
S(dULSLASLVLSO' UA V$SdubSLashso’'a‘’o’(, dudbSiad: ta:,)o)aVoI TIIVI 
/x XLYOWHW TVEZOTD OLNI WYHDOUd AVULSLOOG qvot ste / 
SANG 

{(90T)GWIL TIVO 

COO | = en 2Ou 

/x SHATHSWHHL ACILNAIGI ANV ATHVLSndd 
SSHIOV OL ANIL S$,Ndd HAHLO MOTIV OL AVIGG AIL ALVAHOD «/ 
‘od 

NTHL @ = G1$ndgsd0T di 

/x Nd GVOTLOOM AHL SI Add AI x/ 

, $3 = WOOT$STALsndd 

6T + WOANSNdISDOT = WANSNdISDOT 

/x dO DXAN THL YOK UTAWAN NdD AHL LNANAMONI x/ 

SWANSNdISDOT = GISNdISDOT 

/x GI Nd TVOIDOT BAS x/ 

‘T + WANSNdUISDOT = TVLOL$NdI° (NdISAVOTLOOGM )ATAVLS$Ndd 

/x ENNOD NdD GYOTLOOL LNAWTYONI x/ 

fd1SNdOSSAHd = AI1$NdD° (WANSNdD$DOT)TICVLS$NdI 

/x ATAVLSNdO NI GI Add TVOISKHd GANOINN LIS x/ 

SONG 

£(TSMOOTSTALSNdIO) LASHIOT ATINM OG 


GOWTAdMOLINI 


OD 
6¢ 


Bf 
4c 
on 


hawt MY) 


Ge 
be 


oe 
CA 


be 
Oo 


NON-MDS CONNECTED BOOTLOAD PROGRAM 


62 


8c 
4c 


NI N MN MN NN MOA 


LOTlAS 


HATIdWOD 98-W/Td 


Figure B-2 (cont'd) 


134 





NOILVTIdWOO 98-W/Td dO ANG 


(S)HOUUT WVHDOUd B 
GVGiuY SUNIT Trt 


aes HVSOG = 
qd6 H6080 = 
ad HO008 = 


it 


icy HAVTO 


aZIS WOVLS WAWIXVW 
AZIS VaUV ATAEVIUVA 
AZ1IS Va LNVLSNOO 

aZIS VdadV adoo 


>NOILVWHOAUNI TTAGOW 


/x GOWSTNAISLINI x/ Sana T 6% 
‘ang 2 6 8F 
‘TO ATIHA OF i Lv 
/x “SAW GQHL OL GHLITNNOD Ndd AVOTLOOM AHL 
NI AINO dOOT NOILONOTXY ALINIANI SALVATHD — WVUDOUd NIVW x/ 
/x GTUNACTIONd LANHYUTLNI ONT x/ SULNISGVOTLOOM ANG < OF 
6(WOOTSTALSNAID S ATEVESNdAIO AISNdISDOT)UAV$dHLSLISLS TIVO Zany 


/x KYOWEW TVGOTD NI WVHDOUd dVULSLOOG OL dwor +*/ 


Loaras 


GOWTAdOLINI 


YATIAMWOD 98-W/ Td 


NON-MDS CONNECTED BOOTLOAD PROGRAM 


Figure B-2 (cont'd) 


135 





BOOTSTRAP PROGRAM LISTING 


APPENDIX C., 


‘ngdO ANd 
‘GUOA (NGAVOHOASSSHOOV) FUVI0Rd 
‘UT ENIOd (SOLVLS'ATIA'NLAV) TNVTOTG 
STVNUTLXA (SOLVLS' NLAVOHOT’S SSTOOV ATIC’ NLAV)TUNGTIONd =NadO 


ANN CW 
im oOn © 


| SER OIOROSC ORGIES HE Se ete ae keke cafe fe fe fee abe ac ake fe fe ae a ae ake Be ae ae ae akc ae fe ae ae ae ae fe ae fe ae ae ate ae he fea oe af ae af ae ae ae ae ae ate fe ate ate / 
[x SGUNTAIONd O/I WALSAS V4SG6 OESI TVNYT LAG x / 
J ee Re EE Hee OE BONO BROCE ee ete ae ate ale ate ae ate ae ate ae ale ae fe ae ake abe af ate ae ake ae ae ae ae Fe ake Ee ae be Be ate fe fe abe Be ake abe ake teak fe ate ate ake / 


6‘ 9639b, ALIVYTLIT XLXVW 
‘ 10, AIIVAGLIT SNOGSdULSSLE FUVTOIG 
STUOM (XHQNI‘d1I‘SO) TUVIONG 
STLAX (IST) TAANISTANYAY 
“CUOM (SNVULSTVASOLVLS  TVASULVLSS TVASOLVLS'NLAVSTANYTY) FTUVIOGG T z 


a4 
te) 


1 se Se 2 tee ae Se ae a te ae he ae ae fe fe ake ae be fe ae ae fe ae ale ae fe abe abe ae ae fe ae ake fe ae a ae af af ae ae fe abe afe ae ae ae hea ae fe ae aa ae abe abe ae oe ate ae fe ate ae // 
/ * SNOILVUYVTIOAG VLiVd TVOOT xe / 


/. ste ste se ste te ste ste ote o8 ake ae ate ate ate ste ate ate ae ate oe oe ate oe ake ate ate te ote ate ate ate ate ale ate oft ae ale ae ote ate ate ake ate fe ate ste afe he afc ate sfc ate Ye ole ate oe fe ate ate aft te ate ate ae ate ate / 


fod :dowsdVudSsLood T 
/* RINGOW dVULSLOOM NIDA x/ 


/. ste ste ste se ste ate ak see ke te 68 AON @2 GT TIa JdaS*°dudbS bd se se ste ae We ste ate se ate ate / 


QMOUVT OYS°du¥LSLA: Ti? GBWId AT CAMOANT YATIAWOO 
do’ dudSha? TH: NI GQHOVId ATAGOW LoOAao 


GOWdVHLSLOOM ATNGOW JO NOILVIIMWOO 2° TA 98-W/Td II-SISI 


dowd VuULSLOOL YATIdWOOD 98-W/Td 


136 








/ ste ste se 3He ake Se ote ae Se Se oe ae ake 8 8c ae ae ae ake fe ofc ate ate ake ofe she ake fe afk ke fe fe ake ae ake ate ae abe ae Be she fe ate ake ake se ate ae le ste ae oie ke ake ake ae ake ake ae ate te ate ae ac ote ate / 


/ x "¥Od TI¥VW S,Ndd GHD NI GQHAITOTE SI qoddT x / 
/ x —-MONYOVY NV TILGNO WOOT NIdS V S&HS LI °ALVLS LIVA SALVAYO x / 
| Ser een en nn rr rrr rrr te / 
[x ; OdOSLIVA xe / 


1) she she he sie ote ote te ote ke cate eke ake 2 ae ake ale ate ate ake ate ake ote a%e ake ate ate ake Ste ake ake ate ate ate afe ate afe we ake ake ate afe ate of ae ae af ate fe ale ole a%e ote ate ake aie ate ake ate ate ate ake we ate te / 


| ste se se ae ae ae ae ae 28 ae ae ke ae ae a oe fe ik ae ae fe aie ae af ae ae ae ae ae ak ae a fe be ole ake ae abe a ae ake abe ae ae ale ae ae ae abe ak ae ae ae ake ae ae fe 98 de abe ae ate ak ate ate / 
/ ve SHUNdHIOdd TVOOT ae / 
/ se se se ah se ae ae ke fe ae ae ae ae ae ae ake ke ae ah ote abe ele ale afc ake fe ae fe ie afc ae fe fe ae ae fe ak ae 86 fe of ae he fe ke ale fe ale abe ae eats abe ake abe ake ae ae ae ae ale ate ate ate ake / 


STTIASXAHSAVGYN ANG 


fYUTENIOd (UbLdSdI‘UdsSSo*UddsSaANd) TUVIOIG 
STVNUGT DXA (ULdSdISHLdSSo*eddsSddNG)AUNACTIONd *ATIASXAUSAVAY 


J ste ste ste se ste se sie te oe ste ste ate ae ae ate ae ake ake ae ake ste she ake ate ofc ote fe ote ale ate ate ale ole ste ate ate afc ae ate ate ate ate fe afe sfe ake ote ate oMe ate ale ate ate ate ate afe ate ate ake ate ate ate ate ate ste ate / 
/ x SHUNGTOOUd TVYNUTLXG x / 
7 ste ste se ste oe se ate ake oe ote fe ake oe ae ke ake alk ake fe fe fe oe ae oe ae fe ate ake ake ake ake ake ate ake ae ake ae ake ake ake ae ole ale ak ae ake fe ake ale att ae 8c ofe ste fe fe fe ate ale ae afe aft ake ake ate ate / 


:LIXG nag 
STVNUGDXY FUNCTOIONUd *LIXT 


-AaSOTO ANG 
SUALNIOd SNLVLS FUVTIEC 
-QHOM NLAV AUVTOIE 

{TVNHGLXA (SOLVES*NLAV)TUNTIIONd 2 HSOTO 


‘qvaud ang 
‘UTLNIOd (SHLVLIS*STVOLIV' Uta and) TAVIORA 
‘TYOM (INNOO‘NLAV) TAVIOIC 
STVNUGTLXA (SHLVLSS TVOLIV' BNNOO ‘Utdd ond’ NLAV) TUNdAIONd pie ' 
a9 


GdoOWd VuULSLOOE HATIdWOD 98-W/Td 


aA NOW AQ ef OQ 


AN AN 


Td 


Og 
6t 


BT 
aT 


oT 
ST 
FL 
ot 


al 
TT 


OT 


137 





/x QUACTIOUd NdOSDIVA ANT x/ SNdOSLIVA ANG 2 ee 
/x ATIHAM OD x/ Sana gc 2 
$9 = YOOTSTELSNndd 3 nS 
‘9NOd = DVIA$Ndd Orcs 

/x WOOT-NIdS LIX 

GNV QNOG Ob DVIdSNd) LAS NEHL DBAS ADAAIMONHOV AIL / 
NTHL ANOAGSHUSSLA = WOVSNdI*(CISNMISLIVM)ATIAVEShdd JI e 62 
J/x DIS SI OVId ADACATMONXOV Odd AI AAS OL WOWHO x«/ 
SCNT b- 86 Bz 
£( T° HOOTSTELSNdIO) SASHOOT ATIHA OG e Die. 
‘GNOG <> DVIMSNdD ATINA OG 2 92 
SGNOGSLON = 9V¥TASNdI 2 G2 
/x NIMS Ob WOOT-NIdS AZITVIGINI x/ 

‘90, ATIVHALIT ANOISLON 

‘ 10, ATIVUGLIT qNod 
“qLAG DVTASNdD TUVIOIG 2 Gee 

‘GLAM ULdSHIOTSTAL GASVA WOOTSTALSNdI 

‘UTLNIOd ULdSYIOISTAL 

‘(ALAC LOLSNdD ‘UTLNIOd TI¥WSNdoO 
‘LAG NWOVSNdD SALAM AI$ndd) 

TUNDONNLS (S)ULdSTALSNdD GASVA ATAVLSNdd 

‘UPLNIOd UbdSTah$Ndd 
‘TLAT GISNMOSLIVA AUVIOAG 2 2 
SENVHGNGAGTH (HbdMSVIOTSTAL YSdSTALSNdID AISNdISLIVA)GTUNATIONd *NdISLIVA Towed 


LOILTS 


qoOWdVuLSLOOd YATIdWOO 98-W/Td 


6 





/x HHNGTIONM GUOMSLNO ANT x/ ‘(QHOMSLNO ANT 
$((Hi2 UNV (ANTVA)MOT)IIOSV)YVHOSLNO TIVO 
‘((HdO GNV (b°(ANTVA)HDIH)YHS)IIOSV) BVHOSLNO TIVO 
‘((Hi@ GNV (ANIVA)HDIH)ILTOSV) AYVHOSLNO TIVO 
‘((Hdd GNV (0° (TNTVA)MOT) HHS) IIOSV)XVHOSDNO TIVO 


‘( dGOqV6829GPo2Ta,) VAVA ALAT (x) IIOSVY FUVIOGG 
‘GHOM ANIVA TUVIONG 
S(9NTVA)THNGTIONd :CHOASLNO 


/ ste ste ste se ste ste se te sie aeste te te se ake fe ae ae ote he ate Se ate ate ve ate ate ofeo'e fe ate fe te te ate ate ae ste ate ate ale ate ake ale ate ate ate ale fe ake ate ate afe ake ate ofc ate ate ake fe af ate ate ate ste ate / 


/ x *“SAaNTIVA oe / 
/x SOLVLS FIVSSHIW YOMUT BNIYd OL TISN °ct/9B IASI] AHL JO 'LYOd xe / 
/ xe O/I IVINS WHHL OL SHYMEWAN SLNdLNO HOIHM GNIDNON AASITISN ve / 
| er er nn ne ne nn nn rrr * / 
/ xe CYOMSLNO x / 


7 rie ote ste af ste ste ote ale ote ate a'¢ ate ake ave she ate sft ate se ate se ste ate ake He te ofe fe ate ale ate te ale ate ste ate he fe ate fe fe ate fe ae ake ae ate fe ake oe ah ote ate Ae ake fe fe ake fe se ate ote ate se ate te / 


/x TUNGTOONd UVHOSLNAO ANT x/ SYVHOSLNO ANG 
SHVHOD = (H8d4)LNdLbNo 
‘aNa $0 = (HIG ANV (HVGG)LNANI)ATIHA OG 


;abAC UVHO AUVTIOI 
§$(YVHO)THNCHIOUd sUVHOSLNO 


yi ate ste ste ste ste aie afc ate sfc oe ae ate ake ate ote ste ote ate ate ale ste ate ate ste ote ake oe ole afe sie ote ote ofe ate she aie six aie ste she ste ate oie sie of af ate ste ate ie ste ote ate ote ste of ate ate ate aie ste ote steste ste ate / 


/ “LUO GWHL NO SHOVSSAW se / 
/ x Onddd BNIYd OL AXSN °2T/98 IASI AHL AO LYOd O/I x / 
/ x IVIUGS FHL OL SUTLOVUVHOD SINdLNO HOIHA GANIGNON JLITIACN td 
I I I sk 
/ x YVHOS ENO xe / 
J ste ste ste te ste ste ste se ate se te ae ate fe ate ote ae ate ate ate ote ate ae ote ate ate ote afc sie oe ate ate ate ate afc aft ste ate ate ate ate ate ate fe ate ale ate ate ate ofe ate ate fe ate a%e aft ate ae ate ae te ate ofe ate ate ate / 

LOADS 


GOWdVULSLOO"E UATIGWOD 98-W/Td 


WOO AO? 


et OQ 


mat OQ OU 0 CQ 


4 
OP 
GY 
vv 
Cv 


ov 
ty 
OV 


6& 
BL 
9¢ 


Ge 
ve 


139 





/x HUNAGTIONd HOUNASMHLSLA ANT x/ SYOUuaSMHLSLA ONG 
§(TVASLVLS)QYOMSELNO TIVO 
/x LUD OL ANTIVA SOALVLS LNdLAO »«/ 
Sang 
$((Z)9SW)YVHOSLNO TIVO 
{G2 00h Oe=) 7 OU 
/x LHD OL QOVSSAW HOUNRT INdLNO x/ 


‘CNG 
§cOSWO = ULdSdSW 
‘ZOSWO) = HLbdsdsSW 
{TOSWO = HLdSdSW 


S00UdSOI ASVO od 
/x GTOVSSHW HOUND TLVIAdOUddVY LOTS «/ 


‘HVS, ATIVUSLIT 8 a7 
‘,Hd0, ATIVUSLIT 40 
= HOUNR TIIA THNUAY ASOTN, )VEVG ALAM (%)CDSW 
HOUND TALIM TANUAM AVAU, \VEVG ALAG (*%)Z9SW 
HOUND FIG TANUTH NAdO, )VLVG ALIA («) TSW 
‘ALAC (T)ULdSOSW ATSVE OSH 
‘HHENIOd ULd$9SW 

‘TLAG Z TUV1OGC 
‘7LAG DOUdSOI 

‘CHOM IVASLVLS TUVIOT 

f‘ENVHLNTTY (OONMSOIIVASLVLS)TUNGTIONd sHOUNASAULSLA 


e —_—~ 
ey, gn 
& 
ou 


x 


ste se ae se ate He he fe ale aie ae ae ae ae oe afc fe ate ake ae ake ae ake ae oe fe a ate ae ke abe ak ak fe fe ale afe ake abe ate fe ae a ae ae fe Se ae af ake ate ae oe ake abe ae abe ake ae ate ak ae ae ate ate ae // 


| x “STUNGTIONd O/I WHLSAS VW4S6 OSI AHL HLIAM ate / 
/x WHIGOUd V SI GHHAHL JI LUO THL OL TOVSSAW NOU NV SiNAdLNO te / 
TE I I NN a II a / 
/ YOUNIS duULSLa ate / 
(scsi ona Fa apecentnnemmmnmene rants ese tenis 
LOTT 


GOWdVHLS LOO HATIdWOO 98-W/Td 


Nw 8D D&D N oD ON 


89 
6% 


BG 
aS 
9G 


GG 
%G 
&G 
aG 
TS 


BG 


6% 
SP 


140 





fod 
NTHL @ = AI$NddsdOT II 
/x Nd) AGVOTLOOM SI SIHL AI x«/ 


‘() GLITIWOO dVULSLOOL, )VEVI ALAM («) DSWAONG 
‘(LAG (WONSNAOSDOTS SNdUISTVLOL'Z) AHVIOIG 


©(ULdSdISONSWAWD) SV 
(qHOM DUS ‘CUOM TIO) TUNLINULS dIsoyH 
‘YTLNIOd ULdSdISONSWAEW TUVIOAG 


STLAL ULMSNIOT$STAL AISVA HIOTSTAL$NdI 
“HPLNIOd YULdSVIOTSTAL 
‘(TDI TVLOLSNdD ‘YMLNIOd TIVWSNdd 
‘TAG WOVSNdO ‘FLAC AI$Ndd) 
GUNLINULS (8)uddsIgLsndd AAYSVA AITVLSNdI 
‘HG LNIOd UbdSTIL$ndd 
‘ADAG AI$SNdI$SDOT AUVIORA 


{LNVULNTTY JITAINd (LMS ROOTS TAL YLdSTILSNdd*AISNdISOOT)ATUNATIONd sdVULS$Lood 


7 ste ste ste ate Ye oe te Ne ste este sie aXe oe ake she ate she afe ate ofe ate he oe ote ake ake ofe ote ake se ae te ate ae fe ae ate He aK afe afe ote ake afc Ke ake ale ale ote ahe oe ae ate abe ake ale ate ae ake att ate ate ate ate ate / 


/x °RYOWHW TYIOT S,Ndd YVINOIGUVd LVHD GQVOT OL WVYDOUd dVuds te / 
/ x -LOOZ IVIOTDN SIHL S€LNITXA (Ndd CVOTLOOM FHL AC AXYTIOUL xe/ 
/x -NOO S¥) ‘NUNL NI ‘NdO HOVE “URINE AYOWHW IVAEOTD V OLNI te / 
- TIId TANYAM FHL SAVOT (9 = XTTVOIDOT) ndd GVYOILOOG AHL x / 

Mem ee ae ne ee = me 5 / 
/ x dVaLSsLoog ate / 
(MES AA SAO AREAS antes t OSD Aes ot I * SENT es 

LOL T 


dowd VYLSLOOd YATIdWOO 98-W/Td 


oo 
G9 


v9 


“9 


cQ 


t9 


141 





$2 = WOOTSTAL$NdO 
ST - TVLOLSNdO°’ (S@)ATAVESNdID = SNdOSTVLOL 
/x GHLNNOD L,NSI dd 
GVOTILOOd OS ANO LOVULANAS AGNV IVLOL Od9 AVOTLOOR woud 
QaaqVOI TOL S$,0d9 ONINIVWHY FO HYAIWNN AHL INIWHTLAG te / 
Sa Nq 
S(LSYIOTSTALSNOdID) LAISNOOT ATIHA OG 
$(dIo0‘ So0° YTTINIS TANYAHO)ATIASXAHSAVAY TIVO 
/x dd AVOTLOOG FHL AO ANOWRW 
IVIOIT OLNI AWYOWAW TVAOTD NI XII XTH TANYAY AHL AVOT x/ 
6(2°TVASOLVLS)HOHUNTSMHLSLA TIVO 
NIHL @ <> TVASOLVLS JI 
S(TVASOLVLSOS NLAVSTAINNTY)ASOTO TIVO 
/x ATI€ XFH TANHYRH ASOTO x/ 
/x AIIHM OC x/ Sang 
‘tT + XDXVW + XHQNI = XSANT 
/ee SSUTHGAY HATANGSTANUTY ALMWH LXAN OL XAAQNI LNAWTHONI x/ 
$( T° IVASULVLS) HOUUTSMTULSLE TIVO 
NIHL 2 <> IVASHULVLS JI 
{( TVASALVLSA' SNVULOS XLXVWS (XTONI)UTTANISTINNTIO' NLAVSTINUAH)AVAN TIVO 
‘Y¥LX¥YW = SNVUL ATIHM OG 
/x TOW ONILVIIGNI $SALIXT 96Ab NVHL SSIT SI GHHNTASNVUL 
SHLAG TIGNN AYOWSW TV€OT9 OLNI STIG XTH TANYRN AVAU x/ 
$T + XLXVW = XRANI 
/x SSTUCAVY ATTANASTANUTY ALIWA LXIN OL XAANT BNAWTYUONT x/ 
S(T IVASHULVIS)YONNTSMHLSLA TIVO 
NIHL @ <> IVASULVLS aI 
PC IVASHEV ESOS SNVULOS XEXVW UTTAINTS TANAT WO SNCAVSTANUTY )AVaN TIVO 
/x UTAITOL AHOWAW TVGOT9 OLNI ATIA XAH TYNYGN GVAN »/ 
‘(A°TVASOLVLS)HOUNASHHLSLA TIVO 
NHL @ <> IVASOLVLS II 
S(LVASOLVLSO‘'O'TS(, XTH°TENUTH: Td?) POSNLDAVSTAINYUAIO)NAdO TIVO 
/x ATLA XXH THINGY NTO x/ 


GOWdVYLSLOOd 


G8 
v8 
8 


mo 


c8 
T8 
08 


64 
84 


Hox tm) Ww 


hiks 
9d 
Gd 
va 


tH wi dH 
142 


od 


C& 
Th 
Od 


69 
89 
49 


MM) WD i) ) Mm 


LOPETS 


YFTIdNOD 98-W/Td 





/x AST x/ Sana 


{(NOOTSTALSNdII0'ATAVESNdIIN' AS) NMOSLIVA TIVO 

/x QI1d XHH TANUay 
SNIGVOT ANOG ZUV S,Ndd UAHLO TIV TILNN YOOT-NIdS ULNA x«/ 
$9 = W9O0TSTEL$SNdD 
STNOCSHULSLE = WOVSNdI’ (AI$NdI$DOT) ATAVLSNdI 
/x ALATHWOO SI DNIGVOT LVYHL Nndd IVOTLOOR OL ALVOIGNI %/ 
faNT 
£(T*HOOTSTELSNdID) LASNIOT ATIHM OG 
$(d10‘'S00‘ ¥adAINIS TAINUANO)ATIASXGHSAVaA TIVO 

/x dd UVINOILYVd SIHL JO 
hKYOWGW TVIOT OLNI AHOWAW TVGO01D NI GIIA XAH TANYDH AVOT x/ 


[0d 
aSTd 
/x dd QVOTLOOM LON AI x/ 
/x AI x/ -CNa 


fINOGSdHULSLA = YOVSNdI’ (DO) ATAVLSndd 
/x ALQOIXT ONY ATIA TINNY 
GHL OL dWhrf OL S,Ndd TIV MOTTV OL OVTIA ADATTIMONXOV 
LIS ATI1A XAH TEaNYGX AHL AAACVOT TAVH S,Ndd TIV YALAV x/ 
/ Od x/ SANG 
(( FOIOTSTAL$ AdID* TIGVLSNAIID ‘WANSNAISDOT)NAMISLIVA TIVO 
/x GTIA XTH TANYA AHS 
ONIGVOI ANOG SI Ndd UAHLO FHL TIGNA YOOT-NIdS UMINA x/ 
$9 = WOOTSTaL$ndd 
SdVUbSSLOOTOD = TIVWSNdd’ (WANSNdIS DOT )ATIVESNdd 
/x WYHDOUd dVULSOOE AHL YAINA OL Ndd MOTTVY OL DVIZ LAS x/ 
SONT 
{( T° WOOTSTAL$NdID) LASHOOT ATIHA OG 
‘sndosTVhLOL OL T = WANSNdISDOT OG 
/x S,Ndd 
ONINIVWHH JO LYOWHW TVOOT OLNI ATIA XXX TANUTX AHL AVOT x/ 


COWdVuLSLOOL 


VOT 
SOT 


M) &%) 


cut 
Tat 


OOT 
66 
86 


em) MH (2 


v6 
c6 


wv a 


a6 
T6 


tH ot 


36 
68 
88 


MHwK 


LOTCAS 


YATIdWOD 98-W/Td 


143 





NOTLVIIdWOO 98-W/Td dO ANG 


(S) 


IOV Haddad = adZi 
qgNtaggt HOc2e = dZI 
Id HA600 = aZI 


a466 HGdAlO 


YOUUM WVHDOUd A 
QVau SANIT 9c 
S: WOVLS WAWIXVW 
S VauV GTEVIeVA 
S Vad¥ LNVLSNOO 
aZIS VWauH¥ adoo 


>NOIGVWHOUNI GIACOW 


/x GOWSKVULSSLOOM ANA x/ SING T ett 
/x TUNGAIONA AVULSSLOOM ANA «/ ‘dVuULSSLOOd Ana Sj ttt 
£(A1$Ndd° (CISNAIISDOTITTAVESNAII*S TISNAISDOT)HLdSdISONSWAW TIVO zZ ott 
/x WYHDOUd ISNHAX OL dwar x/ 
‘dl = ddO°dISox 2 6OT 
$9 = 9NS°dISOX Z SOT 
/x “HULdSdISOYSWHW HAGNIOd AHL ATING OL aasn 
GUY ASTHL “ATIIASXAHSAGVAN NI ATI XH IGNNGN THD WOU 
QULOVULXT THRM HOIHM ‘NVHDOUd TSNYDY YOR SdI GNV SO LHS x/ 
Sana e 23T 
$((Z)DSWONG)HVHOSLNO TIVO S OGT 
‘et OL @ = Z Od cf GOT 
/x LHO Ob TOIVSSAW ALATMIWOO dVULSLOOd LNdLno x«/ ‘ 
LOALT 


qowdVuLSLoog 


YGTIdWOO 98-W/Td 


144 





vt 
eT 
A! 
a 
ot 


fUOUNR ANT 
Sana 
$((X)DSWXYTH)UVHOSLNO TIVO 
‘6T OL A = X OG 
$(, HOUND SATIASXAHSAVAN, )VLVG ALAM (*)OSWXTH TUVIONG 
‘GDA X AUVIONG 
‘TUNAdTIOUd 2: YOuUuT 


MA NNN 1) MON 


SUVHOSLNO ANG 
‘fe = (H8d0)LNdiLno 
(HT@ GNV (HVQG) LAUdNI)ATIHA OC 
‘abla f£ @uvioga 

‘(f)GTUNdTIONd =UVHOSLNO 


= 

= 

= 

SQ 

i 
NOAWORbP OD 


MW QUAN CO 


ot 


0d §==dOWSXdHsavad 
/x G@TNGOW XHH AVAN NIDTE x/ 


7. ste ste she ake ste ake sfc ste ate ste 3% ste fe 98 AON @2 GqTid JUS *XHHCY ste se ahe oe a8 te ake oe ste ate / 


ADMVT OUS*°XHHTNS THs OBWTId AT CHNOANT YaTIAWOD 
PHO’ KXHHaUs THs NI ddovId ATNGOW LOArdo 
COWXHHAVAY FTINAGOW FO NOILVIIMWOO e° TA 98-W/Td II-SISI 


qOWXAHAVa a YWATIdWOD 98-W/Td 


145 





‘quom I 
‘aquom LASIIO 
‘TLAG OVTASXTH 
‘ELAM (HX SL SWOASSHOTHD) 
/ ‘“TLAG (NATSAMALSOTN) TUVIOUG Z 61 


SdUOM HLAUSTOUVSWIW AUAISVG TOUVSCHOMSWAW 
‘TLAC ULASTOUVSWAW GASVA TOUVSWAW 
‘(ULdSTOUVSWAWA) LV 
(THOM DOS “THOM TiO) AYNLONULS TOUV 
‘UTLNIOd ULMUSTOUVSWAW THVIIOGG Z BT 


SquOM Ubd$dI AaSvVa di 
‘URLNIOd UbdsdI 
‘CqHOM ULbd$SO GaSvd Sd 


‘UMENIOd ULdSSO ANVTONG C at 
SHLAG (AOGOT)ULdSTINd GASVA YTAANG 

‘UTLNIOd UbdSding AYVIOUA c OT 

fQITANd (HUbdSdI *ULd$So*addsS TAN) TUNATIOUd = ATIASXAHSAVaAY tT GT 
/ ste ste ste se ste ste ate ste ate ate ate ate fe afe ake ate ahe aft fe ake ate ate ste ate ote ale ate fe ote ate fe ae fe ake ate alec ate ot ate ake ake ak ok ake ae ake ak ae oe ate ote fe ae oe afk ae ae ae ole ae ate ate ae ate / 
/ “URTING AMOWHW IVGOTID FHL NI SQIIG XAH AHL SAVAU LI SV x/ 
/ x TTIG LOTLAO LUWNIG V OL TIA TWWIOGAVXGH AHL SADNVHD ATIA x/ 
/x SXTHSAVEN ATSNOANVLINWIS °“ATIX XHH AHL NI GHIdAIOGdS SASSTUAAY x/ 
/x {HL OL DNIGHOOOV AYOTWAW TIVOOT NI ATIGA XGH AHL SALVIOTAY x/ 
/ x ANY Udind RYOWUW TVEOTS V NI GaLVOOT ATIA IVWWIORGVXGH x/ 
/ x ¥V SAVGuU LI “WVHDOUd dVUYLSLOOT AHL AO LUVd SI AUYNAHIONd SIND x/ 
a x / 
/ x TTIASXTHSAvay % / 

1 se se se shee se ae fe ae ate fee ae ak a8 ae oe fe ake ole fe oe ate afc ate ae ate ake fe ate ate ate ae ae ate ae ae ae ate ake ale ale ate abe ate ale ake ake abe ate ste ate ate ate ae ate ate abe ae ate ate ste ateate ate / 

LOlAS 


GOWXAHAVaY HATIdWOD 98-W/Td 


146 





SG LAGSQVGH ANG 

‘i NYOLIY 

‘Lb + WOSSHOGHI = WNSSHOTHD 

6((UVHISCVTN)XMHSDHI)MOT + (0'L)IHS = Lb 

$((YVHISTVEH) XAHHSDHO)AOT = LE 

ST LAd L SHVIORC 

/x ANIVA ALAT 
LUYNIG HIGHL SNYOLTHY AGNV SALAT XTH OML SAVAN ANIDNOY SIHL «/ 
‘FLAG TUNAGTOOUd *ALACSAVAY 


SHYVHOSAVGH ANG 

SUVHO NHOLAY 

el 

‘(1)uadind = UVHO 

STLAG UVHO TUVIIAA 

/x “TLIG FHL SNUNLAY 
NV SHLAG FO UTAANA AVUNV NV WONT ALAT V SVAN ANIDNOW SIHL x/ 
‘TLAG AUNAGTIONd : UVHOSAVAY 


‘X¥THSDHO ANG 

6(H4G ~ 9) TIGNOd NHNLAN ASTA 

‘(HOS - O)TTANOT NUNLAYX NAHL (6, => 0 dI 

6‘qLid 9 TUVIOIGG 

/x “KLIGITIVA LOMNI YOU AAVW SI WOAUHD ON 

THOGTIOUd AHL JO ANIVA AHL SV SI SNUNLAN UNV LNATVAINOT AUVNIG 
SLI Ob IIOSV¥V WOR HUMLAWVUVd GLNAUNI AHL SLUAANOD ANIGNOU SIHL x/ 
SQYOM (9)MUNGTIONd * XTHSDHOD 


‘TLAG C ANVION 
io XTHSAVAN AAUALNA, )VLVG FTLAG (*)OSWH FTUVIOR 


‘tO, ATIVYILIT ANOGSXAH 
‘ 00, ATITVHALIT ANOGSLONSXAH FHVTIOAC 


TH 
OV 
6 
Be 
ae 
9¢ 


tM) 8 02 wD MD MD 


GE 


ve 
ee 
ok 
ez 
Ve 


9998990 COUN 


62 


8e 
uc 
G2 
ve 


Mmmm N 


Loaras 


GOWXHHAVAY 


YMTIdWOO 98-W/Td 


147 





‘LaSdi0 = dI NUaHL O <> LYSAMO AI oro 
NGHL TO = WdkbSoay Al eg ¢9 
/x QHOODTY JOT AI x/ 
SCUOMSAVEN = DUS TOUV g 29g 
NGHL CO = AdALSOTY AI SS 
/x GHOOTY SSHUACAVY AI x/ 
fang ’ 69 
‘qUOMSAVaUN = dI ’ §@666S 
$Q@HYOMSAVaN = SO >’ 863 
fod ec as 
NTHL C3 = GdALSOMH AI Eg gg 
/x GHOOTY SSTHAAGV LUVLS AI x/ 
‘G7 LIGSQVGN = AdALSOG ge gs 
‘LaSdd0 = JdiO° THUY ec FG 
SCHOMSAVaHN = LASIAIO Canes 
‘a LAGSAVAN = NAT Secs 
/x AKL GHOOWN ANY LASAAOSHLONAL HSITAVGST x/ 
SaNq p 6rtS 
§ 3, <> UVHOSAVGN ATION Od Gg gg 
‘SNOGSXTH <> OVIASXTH AIIHM OG 2 6h 
/x TOU TIGNN ATIC XXH AVAN x/ 
‘FNOCSLONSXAH = OVIASXAH $0 = I 2 Lv 
/x OVTASXAH ANV XMONI UTAMINA AZIIVILINI x/ 
/ vedo teteteok eto wrote WVUDOUd NIVW DLTIASXTHSO VT NI DUE tect ate ste te ete fe oe ke fe oka ake ke / 
‘CHOMSAVEU ANG g QF 
‘(GT DATSAVGN)TTAaNOd + (B°(L)TTIINOAT)THS NYNLAY Oy Se, 
‘G@LAGSaVad = Oo ge th 
‘GqLAd L GUV1IONG c &F 
/x “OOTVA GUOM 
LUVNIG UITHL SNUNDAY GNV SALAC XTH YNOT SAVAN ANIDNON SIHL x/ 
$‘GUOM BUNGTIONd :aquOMSa Vay 2 cP 


LIGlAs 


GOWXTHAVGa A UFTIAMWOO 98-W/1d 


148 





NOILVTIMWOD 98-W/Td JO ANd 


(S)HOUUT WVHDOUd 2 


ade HvVTOO = 
ate HAil@O = 
qd HOO09 = 


a2z9gs RASA’) 


/« QOWSXUHSAVGY »/ 


/x QUAGTOONd GNA wx/ $ATIASXAHSAVEN ANG 
/x QATIHM OG x/ Sang 


SGNOGSXUH = DVTIS XaH 
NGHL @ = NUT ANV 10 = Gdibs$oau al 
/x Q@I1G fO ANAT AI x/ 
‘NOHHUT TIV NAHL @ <> WASSHOGHO II 
‘7 LiGSavad = L 
SONa 
‘[ + aiO°TOaV = AIO’ TOUV 
‘MOUND TIVO NUHL LD <> TOUVSWAW AI 
ST LIGSCVAN = TOUVSWAW' L 
‘NdqT OL T = ¥ Od 
/x UTTINA OLNI VLVA GUOOTH AVAU x/ 
NGHL @O = WdILSOTN AI 
/x QUOOTH VIVA GI x/ 


GOWXGT HAVA 


QVauy SUNIT 621 


aZIS WOVLS WAWIXVW 
aZIS Waa GIAVIdVA 
€AZIS VHUV LNVLSNOO 


aZIS Vaav adoo 
>NOILVYWHOUNT GTINdOW 


wo 


SaNa 08 


64 


82 
hes 
94 


Ve, 
Gh 
Gas 
Ta 
69 
89 
a9 


te modded om OmMrm wn 


99 
LOI’TS 


YFTIdWOD 98-W/Td 


149 





APPENDIX D. KERNEL LOADER LISTING 


J PEAR IE REA FE AE AE HE MEA HK AEA AK AE AE NC AE AK AK A NC AK AEE HE IE AK AK AE AK OK AK AKC AK AE AK 2 AK AE AK AK AR AK AE IRE A ARE A AK OKC A ARIK IK ARE A AEE / 


/* Kernel Loader Foutine 7 
Yi mo en —* / 
/* This pseudo-code is included to familiarize the e/ 
/* reader with the kernel loader routine function and is not */ 
/* tested code. x 


[ RRPE AE MERE EAE HE AE HE AK AK HE IE WK HEA AK AK BK HK AK KE ANE AE AK AK AC AK ANE AK AK BE NE AC AK A AK AE AE AK BK A AE AE AE AK AEA AK I A OK OK KE AK AE NE OK OK IK / 


KERNELSLOADER: PROCEDURE; 


/* SUBROUTINE TO REINITIALIZE THE APPLICATICN PROCESS */ 
REINITIALIZE: PROCEDURE(PROCSNUM) 5; 


/* REINITIALIZE THE ADDRESS SPACE INDEX (ASI) */ 
ASI = &; | 
/* INDEX THROUGH THE PROCESS ADDRESS SPACE (PAS) TO 
RELOAD EACH SEGMENT */ 
DO WHILE( PDT(PROCSNUM).PAS(ASI)<>NULL)OR(ASI <> MAXSSEG)); 
/* RELOAD THE SEGMENT */ 
SEGSLOC = SWAPSIN(PDT(PROCSNUM).PAS(ASI));3 
/* RECORD SEGMENT LOCATION IN THE PROCESS PARAMETER 
BLOCK */ 
PPB(ASI) = SEGSLOC; 
/* INCREMENT. THE ADDRESS SPACE INDEX */ 
ASI = ASI + 13 
END; /* DO WHILE */ 
/* CREATE PROCESS DESCRIPTOR SEGMENT */ 
CALL CREATES PROCESS (@PPB); 


END; /* REINITIALIZE PROCEDURE */ 


/* REINITIALIZE CPU EVENTCOUNT AWAITED VALUE */ 
AWAITSVALUE = 135 


/* ENTER DO FOREVER LOOP */ 
DO WHILE @1;3 


/* CHECK TO SEE [IF THIS 15 THE LOAD CPU */ 
IF LOGSCPUSID = 6 THEN DO; 


/* REINITIALIZE TRE LOAD CPU EVENTCOUNT VALUE AWAITED */ 


CPUSAWAITSVALUE = 1; | 
/* DETERMINE THE NUMBER OF CPUS AVAILABLE FOR RECOVERY 
FROM THE LOAD CPU ENTRY IN THE CONFIGURATION TAELE */ 
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TOTALSCPUS = CONFIGSTABLE(@) .CPUSTOTAL; 
/* INDEX THROUGH THE PDT TO REINITIALIZE ALL PROCESSES a 


DO PROCSNUM = @ TO MAXSPROC; 


/* DETERMINE PROCESS CPU AFFINITY ¥*/ 
PROCSAFFINITY = PDT(PROCSNUM).PCM(TOTALSCPUS ); 
/* IF THE AFFINTIY IS FOR THE LOAD CPU THEN DO */ 
IF PROCSAFFINITY = @ THEN 
/* REINITIALIZE THE APPLICATION PROCESS */ 
CALL RINITIALIZE(PROCSNUM)3 


7/* IF NOT THE LOAD CPU AFFINITY THEN */ 
ELSE DO; 


/* SIGNAL THE TARGET CPU LOADER PROCESS */ 

CALL ADVANCE(SYSSEVCSTEL(PROCSAFFINITY ) )5 

/* ENTER A WAIT STATE UNTIL THE TARGET CPU FAS 
COMPLETED THE PROCESS REINITIALIZATION */ 

CALL AWAIT(SYSSEVCSTEL(G@), CPUGSAWAITSVALUE); 

/* INCREMENT EVENTCOUNT VALUE AWAITED */ 

CPUSSAWAITSVALUE = CPUQSAWAITSVALUE + 13 


END; /* ELSE */ 


END; /* DO */ 


/* RESTART THE SYSTEM */ 

CALL ADVANCE(SYSSEVCSTBL(STARTSEVENT))35 

/* ENTER A WAIT STATE UNTIL RESTARTED */ 
CALL AWAIT(SYSSEVCSTBL(®),CPUGSAWAITSVALUE); 


END; /* IF LOGSCPUSID = @ */ 


/* IF NOT THE LOAD CPU THEN FOLLOW THESE INSTRUCTIONS */ 
ELSE DO; 


/* ENTER A WAIT STATE UNTIL SIGNALLED BY THE LOAT CPU 
TO RELOAD A PROCESS */ 

CALL AWAIT(SYSSEVCSTBL(LOGSCPUSID) ,AWAITSVALUE); 

/* INCREMENT THE EVENTCOUNT VALUE AWAITED */ 

AWAITSVALUE = AWAITSVALUE + 1; 


/* REINIPIALIZE THE APPLICATION PROCESS */ 
CALL REINITIALIZE(PROCSNUM); 
END; /* ELSE */ 
END; /* DO FOREVER */ 


END; /* KERNELSLOADER PROCEDURE ¥/ 
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